Data Engineering Specialist

Job Opportunity

This role is an excellent chance to join a dynamic team and contribute to the development of innovative data solutions.

About the Role

We are seeking a highly skilled Junior Data Engineer to support our data science projects. The successful candidate will design and build data pipelines, utilizing software engineering best practices and modern technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro, Docker, Kubernetes, AWS, GCP, and Azure.

Main Responsibilities

• \
• Design and build efficient data pipelines that meet the requirements of data science projects.\
• Utilize big data platforms and tools for batch and streaming data processing, including Hadoop ecosystem, Spark, and Kafka.\
• Collaborate with cross-functional teams to develop and implement data-driven solutions.\
• Communicate technical ideas effectively, both verbally and in writing, in English and Portuguese.\
• Develop and maintain high-quality data environments both in the Cloud and on-premises.\
• Map data fields to hypotheses, curate, wrangle, and prepare data for use in advanced analytics models.\
• Create and manage scalable data architectures.\
• Continuously learn and adapt to new technologies and methodologies to stay ahead of the curve.\
• Work closely with stakeholders to understand their needs and deliver tailored solutions.\
• Prioritize tasks effectively and manage competing deadlines.\
• Maintain accurate and up-to-date documentation of all systems and processes.\
• Foster a culture of innovation and continuous improvement within the team.\
• Participate in code reviews to ensure high-quality code and adhere to software engineering standards.\
• Identify areas for process improvements and implement changes as needed.\
• Provide guidance and support to junior colleagues to help them grow and develop in their roles.\
• Cultivate strong relationships with colleagues, partners, and clients to drive business outcomes.\
• Suggest and implement process improvements to increase efficiency and productivity.\
• Conduct regular review meetings to discuss progress and set goals.\
• Become familiar with industry trends and emerging technologies to inform strategic decisions.\
• Engage in knowledge-sharing sessions to stay updated on best practices and industry developments.\
• Achieve and maintain certifications related to data engineering, cloud computing, or other relevant areas.\
• Evaluate and recommend new tools and technologies to enhance data engineering capabilities.\
• Lead by example, demonstrating integrity, professionalism, and a commitment to excellence in all interactions.\
• Perform additional duties as required to support business objectives.\
• Support business operations during times of change and uncertainty.\
• Make data-driven decisions to drive business growth and optimize processes.\
• Develop and implement data governance policies and procedures.\
• Enhance collaboration between teams through open communication and shared goals.\
• Deliver presentations to stakeholders, providing insights and recommendations based on data analysis.\
• Drive innovation and creativity throughout the organization.\
• Manage and prioritize multiple projects simultaneously.\
• Analyze complex problems and develop effective solutions.\
• Anticipate and mitigate potential risks to project timelines and budgets.\
• Serve as a subject matter expert for data engineering topics, guiding others in their understanding and application of data engineering principles.\
• Maintain confidentiality and handle sensitive information with discretion.\\