Staff Software Engineer, Inference at Anthropic

Hybrid - Dublin, IE

Apply
More jobs at Anthropic

Anthropic’s Inference team builds the infrastructure that powers Claude, focusing on compute efficiency and research enablement. The Staff Software Engineer, Inference role involves end-to-end responsibility for large-scale distributed systems, intelligent request routing, fleet orchestration across diverse AI accelerators, and high-performance inference optimizations. The position supports both production and research workloads, collaborating with scientists and engineers to deliver reliable, interpretable AI systems worldwide.

Salary

EUR 295,000 - 355,000

Requirements

Skills

  • High-performance, large-scale distributed systems
  • Implementing and deploying machine learning systems at scale
  • Load balancing, request routing, or traffic management systems
  • LLM inference optimization, batching, and caching strategies
  • Kubernetes and cloud infrastructure (AWS, GCP)
  • Python or Rust
  • Significant software engineering experience
  • Results-oriented with a bias towards flexibility and impact
  • Ability to pick up slack beyond job description
  • Interest in machine learning systems and infrastructure
  • Thrives in environments where technical excellence drives business results and research breakthroughs
  • Care about societal impacts of work

Responsibilities

  • Build and maintain critical inference systems serving Claude to millions of users worldwide
  • Optimize compute efficiency across distributed systems and accelerator families
  • Enable breakthrough research by providing high-performance inference infrastructure to scientists
  • Design intelligent routing algorithms that distribute requests across thousands of accelerators
  • Autoscale compute fleet to match supply with demand for production, research, and experimental workloads
  • Build production-grade deployment pipelines for releasing new models
  • Integrate new AI accelerator platforms to maintain hardware-agnostic advantage
  • Contribute to new inference features such as structured sampling and prompt caching
  • Support inference for new model architectures
  • Analyze observability data to tune performance based on real-world workloads
  • Manage multi-region deployments and geographic routing for global customers

Technologies

PythonRustKubernetesAWSGCPLLM inference optimizationBatching strategiesCaching strategiesMulti-accelerator deploymentsDistributed systemsLoad balancing

Share job

See if your resume is ready for this job

See how our AI can optimize your resume and improve your chances for this role.