The Data Engineer, Data Platform role at VTEX focuses on designing and evolving the data infrastructure that powers analytics, AI, and machine learning. The position is mid‑level, requiring strong engineering fundamentals, Python and SQL expertise, and experience with cloud data platforms. Responsibilities include building ingestion pipelines, migrating to a multi‑engine Data Lakehouse, maintaining platform infrastructure on EKS, and optimizing query engines. The role emphasizes AI‑assisted development, clear technical documentation, and ownership from design to production.
Data Engineer, Data Platform at VTEX
Remote - Brasil
More jobs at VTEXRequisitos
Habilidades
- Strong engineering foundation
- Experience designing data platform architecture
- Proficiency in Python and SQL
- Experience with cloud data platforms (AWS preferred; GCP/Azure welcome)
- Data‑driven mindset and ability to define impact metrics
- Excellent communication and technical writing skills
- Proficiency with AI assistants and code‑generation tools
Responsabilidades
- Design, build, and evolve data ingestion pipelines (Kinesis/Firehose to Kafka/AutoMQ-on-EKS)
- Migrate workloads to a multi‑engine Data Lakehouse on Apache Iceberg and Spark (EMR-on-EKS)
- Maintain platform infrastructure on Kubernetes/EKS, focusing on compute efficiency and reliability
- Optimize query and consumption engines such as Trino, Cube, DuckDB, and Athena
- Address platform tech debt, observability, and engineering‑process improvements
- Participate in on‑call rotation to support a platform ingesting billions of events per day
Tecnologias
PythonSQLAWSGCPAzureKinesisFirehoseKafkaEKSApache IcebergSpark (EMR-on-EKS)DockerKubernetesTerraformCDKCloudFormationFlinkTrinoCubeDuckDBAthena
See if your resume is ready for this job
See how our AI can optimize your resume and improve your chances for this role.