This role is a mid‑level Data Engineer on VTEX's Data Platform team, responsible for designing, building, and evolving the data infrastructure that powers analytics, AI, and machine learning across the company. The engineer will own end‑to‑end features, work on ingestion, lakehouse migration, platform infrastructure, query engines, and observability, and collaborate with AI tools to scale impact. The position is remote‑first and offers a range of benefits including profit sharing, insurance, professional development, and flexible work arrangements.
Data Engineer, Data Platform at VTEX
Remote - Brazil
More jobs at VTEXRequirements
Skills
- Strong engineering foundation
- Fast learner with sharp problem‑solving instincts
- Experience designing and building platform‑level data infrastructure
- Proficient in Python and its data‑processing ecosystem
- Proficient in SQL
- Experience with cloud data platforms (AWS preferred; GCP/Azure welcome)
- Understanding of modern data architectures (data warehouses, lakes, lakehouses)
- Knowledge of data modeling and API design
- Experience with AI assistants and code‑generation tools
- Strong technical writing and spec‑driven thinking
- Knowledge of real‑time/streaming data processing (Kafka, AWS Kinesis, Flink)
- Experience with containerization and orchestration (Docker, Kubernetes/EKS)
- Experience with infrastructure‑as‑code (Terraform, CDK, CloudFormation)
- Experience with distributed processing (Spark)
- Familiarity with open table formats (Apache Iceberg, Delta, Hudi)
- Understanding of data governance and security best practices
Responsibilities
- Design, build, and evolve the data infrastructure that powers analytics, AI, and machine learning across VTEX
- Own features end‑to‑end—from ingestion and processing to storage and consumption
- Migrate high‑volume pipelines from Kinesis/Firehose to Kafka/AutoMQ‑on‑EKS
- Migrate workloads to Iceberg and implement Spark (EMR‑on‑EKS) processing
- Build and maintain platform infrastructure (Kubernetes/EKS, compute efficiency, reliability)
- Develop query and consumption engines (Trino, Cube, DuckDB, Athena) and smart routing across them
- Improve platform tech debt, observability, and engineering‑process improvements
- Participate in on‑call rotation to keep the platform ingesting billions of events a day healthy
Technologies
PythonSQLAWSGCPAzureKinesisFirehoseKafkaAutoMQEKSApache IcebergSpark (EMR‑on‑EKS)TrinoCubeDuckDBAthenaDockerKubernetesTerraformCDKCloudFormationFlinkDeltaHudi
See if your resume is ready for this job
See how our AI can optimize your resume and improve your chances for this role.