Data Engineer, Data Lake

4 semanas atrás

Teresina, Brasil Visa Inc. Tempo inteiro

What you\'ll do Data Teams play a central role at Pismo, providing data-intensive analytics for internal product teams, while also processing and delivering TBs of high-quality data, mainly in streaming form, needed by our partners to produce outstanding products downstream. Operating mainly in Brazil, the data team recently extended its scope to accelerate the platformization of data services globally, with representatives in Europe and soon in the US and Asia. We are looking for talented engineers who share our culture of trust, mutual responsibility, and technological excellence to be part of this unprecedented and diverse global data organization. Serve as a senior technical contributor on Pismo’s data stack, leading the design and development of robust, scalable data pipelines using modern Big Data frameworks. Drive continuous improvement in platform performance, reliability, and cost-effectiveness, proactively identifying opportunities for innovation. Take ownership of production operations and monitoring (file-based systems), ensuring adherence to SLAs and applying best practices for observability and resilience. Collaborate with cross-functional teams to architect, build, and evolve high-complexity data features, reducing technical debt and championing engineering excellence. Provide mentorship to peers through code reviews, feedback, and support, fostering an environment of knowledge sharing and professional development. Ensure that data pipelines follow the best testing, monitoring, observability, and security practices. Participate in on-call rotations, promptly addressing and resolving production incidents, while proactively implementing preventative measures. Build and optimize ETL/ELT processes to handle large-scale data, leveraging advanced data processing and distributed systems techniques. Maintain rigorous data governance and platform standards, working closely with data stakeholders to ensure consistent, high-quality data assets. Conduct root-cause analyses for critical issues, challenging and refining team processes to enhance efficiency and reliability. Influence multiple teams by promoting a culture of technical excellence, forward-thinking solutions, and continuous improvement. This is a remote position. A remote position does not require job duties be performed within proximity of a Visa office location. Remote positions may be required to be present at a Visa office with scheduled notice. Basic Qualifications 2+ years of relevant work experience and a Bachelors degree, OR 5+ years of relevant work experience B1 English or Above (Intermediate) Strong experience as a Data Engineer Good experience with AWS services (e.g., S3, DMS, Lambda, Kinesis, and IAM) Apache Spark experience or similar tools (e.g., Flink, Hadoop) Monitoring tools: Grafana and CloudWatch (logs/tracing/spans/monitoring/alerts/dashboards) SQL experience Desirable Qualifications CI/CD experience Airflow experience Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law. #J-18808-Ljbffr

  • Data Warehouse Architect

    Há 4 horas

    Teresina, Brasil AVM Consulting Inc Tempo inteiro

    Databricks Data Warehouse Architect About the role: One of the largest companies in the world in the Gaming industry is seeking a hands-on Data Architect with DataWarehouse Engineer expertise in Databricks (DBX)and AWS-native data services to spearhead the design and implementation of a new data warehouse instance for a major product line. This role will...

  • Data Engineer

    2 semanas atrás

    Teresina, Brasil Avra Tempo inteiro

    A Avra é uma plataforma de inteligência de dados deep tech, que traduz a complexidade das pequenas e médias empresas em decisões estratégicas para grandes corporações. Desenvolvemos nossos próprios modelos fundamentais do zero — sem depender de soluções de terceiros — para entregar insights inovadores que impulsionam grandes bancos e fintechs...

  • ML Data Pipeline Engineer

    Há 2 dias

    Teresina, Brasil Prosigliere Tempo inteiro

    We're seeking a Data Pipeline Engineer to own and evolve our exercise recognition training data infrastructure. You'll manage the end-to-end pipeline that collects, synchronizes, validates, and prepares IMU sensor and video data for ML model training. This role combines systems engineering, data quality automation, and hands-on problem-solving in a...

  • Senior Data Engineer

    3 semanas atrás

    Teresina, Brasil Swapcard Tempo inteiro

    Our Mission Swapcard is the leading AI-powered event platform designed to drive revenue growth and foster meaningful connections at in-person and hybrid events. We recognize the importance of teamwork in successful events; that's why Swapcard is fueled by a team of innovators who are passionate about helping organizers build future-proof events. Our Vision...

  • Data Engineer | Specialist

    3 semanas atrás

    Teresina, Brasil Compass UOL Tempo inteiro

    Job description . Main responsibilities Criar pipeline de dados para obtenção dos dados com airflow/spark; Implementar controles de acesso e auditoria para proteger dados sensíveis e garantir conformidade com regulamentações; Migrar e integrar dados existentes na nova plataforma, assegurando a continuidade das operações e a integridade dos dados;...

  • QA / Quality Manager- Automation ( Remote )

    3 semanas atrás

    Teresina, Brasil Grupo Data Tempo inteiro

    Hi! We are Group DATA and we are searching for the best talent! Our goal is to simplify our clients' lives with innovative IT solutions. We operate at global scale and we are expanding to Portugal! If you are passionate and have the desire to make the difference, we want to get to know you! Join us to be part of this incredible adventure! Who are we looknig...

  • Semi Senior Etl

    2 semanas atrás

    Teresina, Brasil BairesDev Tempo inteiro

    At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide. Semi Senior ETL / Data...

  • SAP Project Manager SR

    2 semanas atrás

    Teresina, Brasil NTT DATA Business Solutions Tempo inteiro

    Nós nos transformamos. Soluções SAP® em Valor A NTT DATA Business Solutions está há 30 anos no mercado mundial. Somos uma empresa apaixonada por transformação. E queremos transformar o mundo e as pessoas através das soluções que criamos juntos com nossos parceiros. Aqui projetamos, implementamos, gerenciamos e potencializamos continuamente as...

  • Senior Data Analyst

    2 semanas atrás

    Teresina, Brasil BairesDev Tempo inteiro

    At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide. Senior Data Analyst at...

  • Data Scientist

    3 semanas atrás

    Teresina, Brasil BairesDev Tempo inteiro

    Data Scientist - Remote Work Join to apply for the Data Scientist - Remote Work role at BairesDev . At BairesDev, we lead technology projects for over 15 years, delivering cutting‑edge solutions to giants such as Google and innovative startups in Silicon Valley. Our diverse team of 4,000+ top‑1% tech talent works remotely on roles that drive significant...