EmergingAlignment and scalable oversight

Reward Hacking Researcher

Also called Specification Gaming Researcher

A Reward Hacking Researcher finds and studies cases where models exploit the reward or evaluation signal instead of doing the intended task, then tests fixes. It matters because a model that scores well for the wrong reasons fails once it is deployed.

What they do

Start by building environments and evaluations where a model can score well without doing what we meant. Train or prompt policies against them and watch how they cheat. Write up each failure as a clear, repeatable demonstration. Then design mitigations, such as better reward signals, adversarial training, tighter evaluation design or monitoring, and test them on held-out cases. Much of the day is experiment design, reading transcripts and checking whether a fix removed the behaviour or only moved it. Findings go back to the teams who train models and write evaluations.

What a strong hire delivers

In the first months, a strong hire gives the team a small set of reproducible failure demonstrations that people can actually run. They map where current rewards and evaluations are most exposed. They test at least one mitigation against unseen cases and report honestly what held and what did not. They also leave behind experiment tooling that others can reuse.

How you can tell they are good

Ask them to walk through a case where a model gamed a signal, and how they confirmed it was gaming and not a bug. Strong people separate the symptom from the cause and show their controls. Ask how they know a mitigation generalises. Be wary of anyone who only lists known examples, or who claims a fix without testing on unseen cases.

Why it is hard to fill

The work is being done now, but rarely under this title. It sits inside alignment, evaluation, preference learning and training teams, so job searches miss it. Few people have both the research instinct to design a clean experiment and the engineering skill to run adversarial training at scale. Candidates often come from adjacent fields and describe their work differently, so titles tell you little.

Where to find them

Look at people in technical safety research, preference learning and reinforcement learning from feedback, and those building model evaluations. Many sit in frontier labs, safety-focused research organisations, academic groups working on specification problems and applied teams that train their own models. Job titles include research scientist, research engineer, evaluation lead and alignment researcher. Ask about the work, not the label.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Reward Hacking Researcher

  1. Describe a time you found a model exploiting a reward or evaluation signal. How did you confirm it was gaming?

    A strong answer names the signal, the shortcut and the checks that ruled out a bug or noise. They show a clear control or ablation. Worry if the story is anecdotal, relies on reading one transcript, or never tested the explanation.

  2. How would you design an environment that reliably surfaces specification gaming without handing the model an obvious trick?

    Good answers cover a plausible gap between proxy and intent, varied tasks, and a way to measure true task success separately. They consider how realistic the exploit is. Worry about contrived traps or no independent success measure.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are looking for a researcher to find and study cases where our models exploit reward or evaluation signals instead of doing the intended task. You will build demonstrations, test mitigations and help the teams who train and evaluate our models make better decisions.

What you will do

  • Design environments and evaluations that expose specification gaming
  • Produce clear, repeatable demonstrations of failures
  • Develop and test mitigations, including adversarial training
  • Measure divergence between reward and intended task success on unseen cases
  • Review model transcripts to find new exploit patterns
  • Share findings and recommendations with training and evaluation teams

What you will bring

  • Demonstrated experimental work on reward models, preference learning or safety research
  • Experience designing evaluations and controls that hold up to scrutiny
  • Strong skill in training and running experiments on language or agent models
  • Clear writing that others can act on
  • Sound judgement about what a result does and does not show

Nice to have

  • Experience with adversarial training or red teaming
  • Published or shared work on specification gaming
  • Experience building reusable experiment tooling

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Alignment researchAdvancedStudy methods to make behaviour meet intended objectives
  • Preference learningAdvancedLearn behaviour from comparisons and reward signals
  • Model evaluationWorkingMeasure capability and failure modes credibly
  • Experimental designWorkingChoose comparisons that identify credible effects

Other roles in alignment and scalable oversight

Hiring nowScalable Oversight ResearcherA scalable oversight researcher designs and tests ways to supervise AI systems on tasks too hard for a person to judge easily. The work shows…Hiring nowModel Behaviour ResearcherA Model Behaviour Researcher studies how a model's behaviour shifts across contexts and training, and shows what caused the shift. That gives the…Hiring nowModel Behaviour and Character DesignerA Model Behaviour and Character Designer defines and tunes how a model reasons, communicates and handles values trade-offs. They make the product's…

Related roles in other families

Hiring nowPost-Training Research EngineerPost-training and preference optimisationA Post-Training Research Engineer adapts pretrained models into ones that behave as the company needs. They turn preference data and reward signals…Hiring nowReward Modelling ResearcherPost-training and preference optimisationA reward modelling researcher builds models that learn what good output looks like from preference comparisons. They give post-training teams a…Hiring nowReinforcement Learning Research EngineerPost-training and preference optimisationA Reinforcement Learning Research Engineer improves how a model behaves by training it against interactive signals. They matter because reward gains…Hiring nowModel Evaluation EngineerCapability evaluation and benchmarkingA Model Evaluation Engineer builds the tests that show what an AI system can do and where it breaks. Without them, a team cannot tell whether a…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Alignment and scalable oversightAll new job families

Start a search

Hiring a Reward Hacking Researcher?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.