Big Data Expert

JD : We are seeking an experienced Big Data Expert to design and implement scalable data solutions that enable data-driven decision-making across the organization. This role focuses on architecting end-to-end big data pipelines, ensuring data quality, governance, and efficient processing of large datasets. More focused on Data Architectures (Databricks, AWS (Redshift, Dynamo, etc.)), Data Governance, and Data Management. An emphasis on client facing workshops, solution sessions, capability presentations, etc. Candidate should be strong communicators, ideally have worked in some presales / solutioning type team in IT services provider, and are based either in USA or Brazil. Responsibilities: Big Data Architecture: Design and implement scalable, distributed data processing systems using big data technologies (e.g., Hadoop, Spark). Data Pipelines: Build and optimize ETL/ELT pipelines to handle large-scale data ingestion, transformation, and storage. Data Governance: Establish data governance frameworks, including policies for data security, privacy, and compliance. Quality Control: Develop and enforce data quality standards, leveraging tools to monitor and ensure data accuracy and consistency. Cloud Integration: Design big data solutions on cloud platforms (AWS, GCP, Azure), leveraging cloud-native tools. Collaboration: Work with data engineers, analysts, and business stakeholders to align data architecture with organizational goals. Innovation and Optimization: Stay updated on big data technologies and optimize systems for performance, scalability, and cost-efficiency. Required Skills: Big Data Expertise: Hands-on experience with Hadoop, Spark, Kafka, and other big data frameworks. Data Governance: Knowledge of governance frameworks and tools like Collibra, Alation, or similar. Quality Control: Proficiency in implementing data quality measures and tools (e.g., Apache Griffin, Talend, or Informatica). Cloud Platforms: Experience with cloud-based data solutions (BigQuery, AWS EMR, Dataproc). Programming Skills: Proficiency in Python, Java, or Scala for data processing. Database Knowledge: Strong understanding of SQL and NoSQL databases. Problem-solving: Strong analytical skills for troubleshooting and optimizing complex data architectures. Preferred: Certifications in big data or cloud technologies (e.g., GCP Data Engineer, AWS Big Data Specialty). Experience with MLOps pipelines and integrating AI/ML workflows with big data systems. Knowledge of metadata management and data lineage tools. Familiarity with GDPR, CCPA, and other data privacy regulations.