Senior Software Engineer, Inference at Anthropic

Hybrid - Dublin, IE

Apply
More jobs at Anthropic

Anthropic is building reliable, interpretable, and steerable AI systems. The Inference team builds and maintains critical systems that serve Claude to millions of users worldwide. The role focuses on maximizing compute efficiency and enabling breakthrough research by providing high-performance inference infrastructure across diverse AI accelerators. Responsibilities include designing intelligent routing algorithms, autoscaling compute fleets, building deployment pipelines, integrating new accelerator platforms, contributing inference features, supporting new model architectures, analyzing observability data, and managing multi-region deployments.

Salary

EUR 235,000 - 295,000

Requirements

Skills

  • High-performance, large-scale distributed systems
  • Implementing and deploying machine learning systems at scale
  • Load balancing, request routing, or traffic management systems
  • LLM inference optimization, batching, and caching strategies
  • Kubernetes and cloud infrastructure (AWS, GCP)
  • Python or Rust
  • Significant software engineering experience, particularly with distributed systems
  • Results-oriented, with a bias towards flexibility and impact
  • Pick up slack, even if it goes outside your job description
  • Want to learn more about machine learning systems and infrastructure
  • Thrive in environments where technical excellence directly drives both business results and research breakthroughs
  • Care about the societal impacts of your work
  • Bachelor's degree or equivalent combination of education, training, and/or experience

Responsibilities

  • Build and maintain critical systems that serve Claude to millions of users worldwide
  • Design intelligent routing algorithms that optimize request distribution across thousands of accelerators
  • Autoscale compute fleet to dynamically match supply with demand across production, research, and experimental workloads
  • Build production-grade deployment pipelines for releasing new models to millions of users
  • Integrate new AI accelerator platforms to maintain hardware-agnostic advantage
  • Contribute to new inference features such as structured sampling and prompt caching
  • Support inference for new model architectures
  • Analyze observability data to tune performance based on real-world production workloads
  • Manage multi-region deployments and geographic routing for global customers

Technologies

PythonRustKubernetesAWSGCP

Share job

See if your resume is ready for this job

See how our AI can optimize your resume and improve your chances for this role.