Big Data Engineer

Descrição: Big Data Engineer
Our Client is a US technology company looking for a highly motivated Data Engineer with a passion for data, to build and implement data pipelines in cloud technologies, including SnapLogic and AWS.
Responsibilities:
● Develops and maintains scalable data pipelines in SnapLogic and builds out new ETL and API integrations to support continuing increases in data volume and complexity.
● Develop and maintain data models for core package application and reporting databases to describe objects and fields for support documentation and to facilitate custom application development and data integration.
● Monitoring execution and performance of daily pipelines, triage and escalate any issues.
● Collaborates with analytics and business teams to improve data models and data pipelines that feed business intelligence tools, increasing data accessibility and fostering data-driven decision-making across the organization.
● Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes.
● Writes unit/integration tests, contributes to engineering wiki, and documents work.
● Performs data analysis required to troubleshoot data-related issues and assist in the resolution of data issues.
● Work within AWS/Linux cloud systems environment in support of data integration solution
● Works closely with a team of frontend and backend engineers, product managers, and analysts.
● Teamwork: Collaborate with team members. Share knowledge, provide visibility into personal accomplishments and follow directions when provided.
Experience Required or equivalent
● Experience with SnapLogic including writing pipelines that include mappers, gates, logging, bulk loads, and Salesforce SOQL queries;
● Experience with AWS services including but not limited to S3, Athena, EC2, EMR, Glue;
● Ability to solve any ongoing issues with operating the cluster;
● Experience with integration of data from multiple data sources;
● Experience with various database technologies such as SQLServer, Redshift, Postgres, RDS;
● Experience with one or more of the following data integration platforms: Pentaho Kettle, SnapLogic, Talend OpenStudio, Jitterbit, Informatica PowerCenter, or similar;
● Knowledge of best practices and IT operations in an always-up, always-available service;
● Experience with or knowledge of Agile Software Development methodologies;
● Excellent problem solving and troubleshooting skills;
● Excellent oral and written communication skills with a keen sense of customer service;
● Experience with collecting/managing/reporting on large data stores;
● Awareness of Data governance and data quality principles;
● Well versed in Business Analytics including basic metric building and troubleshooting;
● Understand Integration architecture: application integration and data flow diagrams, source-to-target mappings, data dictionary reports;
● Familiar with Web Services: XML, REST, SOAP;
● Experience with Git or similar version control software;
● Experience with integrations with and/or use of BI tools such as GoodData (Prefered), Tableau, PowerBI, or similar;
Database experience:
● Broad experience multiple RDBMS: MS SQLServer, Oracle, MySQL, PostgreSQL, Redshift;
● Familiarity with SaaS/cloud data systems (e.g. Salesforce);
● Data warehouse design: star-schemas, change data capture, denormalization;
● SQL/DDL queries/Tuning techniques such as indexing, sorting, distribution;
Education, Experience, and Licensing Requirements:
● BS or MS degree in Computer Science or a related technical field;
● 3+ years of Data Pipeline development such as SnapLogic (preferred) or Datastage,Informatica or related experience;
● 3+ years of SQL experience (No-SQL experience is a plus);
● Experience designing, building, and maintaining data pipelines;
Where: home -office
Salary: in US$
Re