Staff Software Engineer, Inference at Anthropic

Hybrid - London, UK

Apply
More jobs at Anthropic

The Inference team at Anthropic is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. As a Staff Software Engineer on this team, you will work end-to-end, identifying and addressing key infrastructure blockers while maximizing compute efficiency and enabling breakthrough AI research. The role involves designing intelligent routing algorithms, autoscaling compute fleets, building deployment pipelines, integrating new AI accelerator platforms, and managing multi-region deployments across cloud platforms.

Salary

GBP 325,000 - 390,000

Requirements

Skills

  • Bachelor’s degree or an equivalent combination of education, training, and/or experience
  • Significant software engineering experience, particularly with distributed systems
  • Experience with high-performance large-scale distributed systems
  • Experience implementing and deploying machine learning systems at scale
  • Experience with load balancing, request routing, or traffic management systems
  • Experience with LLM inference optimization, batching, and caching strategies
  • Experience with Kubernetes and cloud infrastructure (AWS, GCP)
  • Proficiency in Python or Rust
  • Results-oriented mindset with a bias towards flexibility and impact
  • Ability to pick up slack, even if it goes outside your job description
  • Willingness to learn more about machine learning systems and infrastructure
  • Thrive in environments where technical excellence directly drives business results and research breakthroughs
  • Care about the societal impacts of your work

Responsibilities

  • Build and maintain critical systems that serve Claude to millions of users worldwide
  • Identify and address key infrastructure blockers end-to-end
  • Maximize compute efficiency for explosive customer growth
  • Enable breakthrough research by providing high-performance inference infrastructure to scientists
  • Tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware
  • Design intelligent routing algorithms that optimize request distribution across thousands of accelerators
  • Autoscale the compute fleet to dynamically match supply with demand across production, research, and experimental workloads
  • Build production-grade deployment pipelines for releasing new models to millions of users
  • Integrate new AI accelerator platforms to maintain hardware-agnostic competitive advantage
  • Support inference for new model architectures and contribute to new inference features
  • Analyze observability data to tune performance based on real-world production workloads
  • Manage multi-region deployments and geographic routing for global customers

Technologies

KubernetesAWSGCPPythonRustLLM inference optimizationBatchingCachingDistributed systemsHigh-performance computing

Share job

See if your resume is ready for this job

See how our AI can optimize your resume and improve your chances for this role.