Anthropic is looking for an Engineering Manager to lead the Agent Prompts & Evals team, responsible for building and owning the evaluation platform, system prompt infrastructure, and supporting model launches, while recruiting and guiding engineers across product and research.
Engineering Manager, Agent Prompts & Evals at Anthropic
Hybrid - San Francisco, CA, USA
More jobs at AnthropicSalary
USD 320,000 - 405,000
Requirements
Skills
- 8+ years in software engineering with 3+ years managing engineering teams
- Track record of building tooling and processes that enable other teams to succeed without needing detailed technical knowledge
- Comfort managing a team with a mixed charter of platform ownership, service to other teams, and launch‑driven operations
- Sufficient technical depth to engage on system design, review pipeline architecture, and debate with senior individual contributors
- Product mindset and willingness to wear multiple hats when the work calls for it
- Demonstrated ability to build and maintain peer relationships with partner organizations that have different cultures and incentives
- Experience recruiting and closing senior individual contributors in a competitive market
- Prior exposure to LLM evaluation, ML experimentation platforms, or model quality work (even tangentially)
- Experience with A/B testing infrastructure, feature flagging, or gradual rollout systems
- Background in developer tools, CI/CD platforms, or testing infrastructure at scale
- History of managing teams that sit between two larger orgs and making that position an asset
- Interest in AI safety and alignment (not required but desirable)
Responsibilities
- Lead and grow a team of prompt engineers and platform software engineers
- Own the product‑side eval platform, including frameworks, dashboards, bulk runners, and CI integrations used by product teams to measure Claude’s behavior and catch regressions
- Own system prompt infrastructure for versioning, deployment, rollback, and review tooling across claude.ai, the API, and agentic surfaces
- Be the steady hand through model launches, acting as the backstop during high‑stakes operational moments
- Build durable collaboration with other eval groups across the company, owning shared roadmaps and avoiding tragedy‑of‑the‑commons on shared infrastructure
- Recruit, close, and retain engineers who want to work at the intersection of product engineering and model behavior
- Shape where the team invests next by sequencing frontier eval development, launch automation, and deeper prompt engineering support
- Push the team toward measuring hard, high‑impact metrics such as behavioral drift, prompt quality, and harness parity
Technologies
CI/CDA/B testingFeature flaggingGradual rollout systemsDevtoolsTesting infrastructure
See if your resume is ready for this job
See how our AI can optimize your resume and improve your chances for this role.