As a Research Lead on the Training Insights team at Anthropic, you will develop the strategy for measuring and characterizing model capabilities across training and deployment. This hands‑on leadership role involves driving original research into new evaluation methodologies while leading a small team of researchers and research engineers. Your work will span the full lifecycle of model development, including building new long‑horizon evaluations, developing novel approaches to measuring emerging capabilities, and deepening our understanding of how capabilities develop during production RL training and beyond. You will collaborate across teams such as Reinforcement Learning, Pretraining, Inference, Product, Alignment, and Safeguards to map the landscape of model evaluations and identify critical gaps. This role carries significant visibility and impact, shaping the evaluation narrative for model releases and directly influencing how Anthropic communicates its models internally and externally.
Research Lead, Training Insights at Anthropic
Remote - Remote, United States
More jobs at AnthropicSalary
USD 850,000 - 850,000
Requirements
Skills
- Significant experience designing and running evaluations for large language models or similar complex ML systems
- Led technical projects or teams, either formally or through sustained ownership of critical research directions
- Equipped to design experiments and write code—can move between research and implementation fluidly
- Strategic thinking about what to measure and why, not just how to measure it
- Ability to synthesize information across multiple teams and workstreams to form a coherent picture of model capabilities
- Clear communication of complex technical findings to both technical and non-technical audiences
- Results‑oriented and able to thrive in fast‑paced environments where priorities shift based on research findings
- Deep care about AI safety and a desire to directly influence how capable AI systems are developed and deployed
- Experience building evaluations for long‑horizon or agentic tasks
- Deep familiarity with Reinforcement Learning training dynamics and how model behavior changes during training
- Published research in machine learning evaluation, benchmarking, or related areas
- Experience with safety evaluation frameworks and red teaming methodologies
- Background in psychometrics, experimental psychology, or other measurement‑focused disciplines
- Track record of communicating evaluation results to inform high‑stakes decisions about model development or deployment
- Experience managing or mentoring researchers and engineers
- Bachelor’s degree or an equivalent combination of education, training, and/or experience
Responsibilities
- Build new novel and long‑horizon evaluations
- Develop novel measurement approaches for understanding how model capabilities emerge and evolve during RL training
- Lead strategic evaluation coverage across the company
- Shape the evaluation narrative for model releases
- Lead and mentor a small team of researchers and research engineers, setting research direction and fostering a culture of rigorous, creative research
- Design evaluation frameworks that balance scientific rigor with the practical demands of production training schedules
- Build and maintain relationships across Anthropic’s research organization to ensure evaluation insights inform training and deployment decisions
- Contribute to the broader research community through publications, open‑source contributions, or external engagement on evaluation best practices
See if your resume is ready for this job
See how our AI can optimize your resume and improve your chances for this role.