Senior Software Engineer, Inference at Anthropic

Hybrid - Dublin, IE

Apply
More jobs at Anthropic

We are looking for a Senior Software Engineer in the Inference team to build and maintain critical systems that serve Claude to millions of users worldwide. The role involves designing intelligent routing algorithms, autoscaling compute fleets, building deployment pipelines, integrating new AI accelerator platforms, contributing to inference features, supporting new model architectures, analyzing observability data, and managing multi-region deployments. The team focuses on maximizing compute efficiency and enabling breakthrough research, tackling distributed systems challenges across multiple accelerator families and emerging AI hardware across cloud platforms. The position is hybrid, based in Dublin, and offers competitive compensation and benefits.

Salary

EUR 235,000 - 295,000

Requirements

Skills

  • High-performance, large-scale distributed systems
  • Implementing and deploying machine learning systems at scale
  • Load balancing, request routing, or traffic management systems
  • LLM inference optimization, batching, and caching strategies
  • Kubernetes and cloud infrastructure (AWS, GCP)
  • Python or Rust

Responsibilities

  • Designing intelligent routing algorithms that optimize request distribution across thousands of accelerators
  • Autoscaling the compute fleet to dynamically match supply with demand across production, research, and experimental workloads
  • Building production-grade deployment pipelines for releasing new models to millions of users
  • Integrating new AI accelerator platforms to maintain our hardware-agnostic competitive advantage
  • Contributing to new inference features (e.g., structured sampling, prompt caching)
  • Supporting inference for new model architectures
  • Analyzing observability data to tune performance based on real-world production workloads
  • Managing multi-region deployments and geographic routing for global customers

Technologies

KubernetesAWSGCPPythonRust

Share job

See if your resume is ready for this job

See how our AI can optimize your resume and improve your chances for this role.