Anthropic’s inference fleet powers Claude across millions of users. The Inference System Dynamics team analyses and optimizes throughput, latency, reliability, and correctness across the entire stack, from accelerator kernels to distributed routing and autoscaling. The role involves cross-layer investigations, building observability tools, improving correctness evaluation pipelines, and collaborating with kernel, serving, and autoscaling teams to prioritize high-impact optimizations.
Performance Engineer, Inference Systems en Anthropic
Híbrido - San Francisco, CA
Más vacantes en AnthropicSalary
USD 350,000 - 850,000
Requirements
Skills
- Performance engineering experience: profiling, roofline analysis, latency/throughput optimization, and root-cause investigation in complex production systems
- Proficiency in Python
- Solid data analysis skills (SQL, pandas, or similar)
- Ability to communicate quantitative results clearly in writing
- Interest in correctness as an engineering discipline: numerics, evaluation design, regression detection
- Bachelor’s degree or equivalent combination of education, training, and/or experience
- Experience with ML systems, especially inference infrastructure or LLM serving stacks
- Familiarity with GPU/TPU/accelerator performance concepts (memory bandwidth, kernel overheads, quantization, collective communication)
- Experience with reliability engineering for high-throughput services (autoscaling, load balancing, request routing, tail latency)
- Experience with model evaluation or numerical regression-detection pipelines
- Experience building observability or telemetry for distributed systems
- Comfortable influencing through evidence rather than direct ownership
Responsibilities
- Run cross-layer performance investigations across throughput, latency, and reliability, sizing the gap between actual fleet performance and theoretical rooflines, identifying root causes, and quantifying the value of closing them
- Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations, and lead the investigation when it catches a regression
- Build observability dashboards and modeling tools that make throughput, latency, cost, reliability, correctness, and their interactions legible across the stack
- Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land the highest-impact optimizations
- Stack-rank a large surface area of opportunities by impact and effort, and say no to the ones that don’t make the cut
Technologies
PythonSQLpandasGPUTPU
Descubre si tu currículum está listo para esta vacante
Mira cómo nuestra IA puede optimizar tu currículum y aumentar tus chances en este puesto.