Hiring nowPost-training and preference optimisation

Reward Modelling Researcher

Also called Reward Model Scientist

A reward modelling researcher builds models that learn what good output looks like from preference comparisons. They give post-training teams a signal they can trust, and show where that signal can be gamed.

What they do

They train reward models on pairs of outputs where one was preferred, then test what those models have actually learned. Day to day that means designing experiments, checking behaviour on data the model has not seen, and probing for shortcuts such as length bias or style bias. They study how noisy or conflicting preferences weaken the signal. They work closely with whoever runs reinforcement learning, because a reward model is only useful if a policy cannot easily exploit it. They write up findings, keep runs well tracked and decide when a model is good enough to use.

What a strong hire delivers

A reward model the post-training team is willing to optimise against, with a written analysis of its biases and the ways it can be exploited. Clear evaluation on independent comparisons. A repeatable training and testing set-up that others can run. Early warning when a policy starts gaming the signal, and a clear view of what to fix next.

How you can tell they are good

Ask them to walk through a reward model they built and how they knew it worked. Strong people talk about agreement on comparisons the model never trained on, and about what they did when a policy found a loophole. They explain preference noise without hand waving, and they are open about where their own model was weak.

Why it is hard to fill

The title is rare and inconsistent. The same work appears under preference modelling, alignment research or post-training research. Few people have seen a reward model get exploited in a real training run, and that experience is what matters. Many candidates are strong on fine-tuning or reinforcement learning but have never studied the reward signal itself. Those with both usually sit inside a small number of teams that are reluctant to lose them.

Where to find them

Look at research scientists and engineers in post-training teams at model providers and frontier labs, and at well-funded start-ups that fine-tune their own models. Alignment and safety researchers often have the right instincts. So do reinforcement learning researchers who moved toward language models, and academics working on preference learning, robustness or evaluation methodology.

Indicative pay

United Kingdom£90k to £200kBase salary, 0 published sources
United States$200k to $500kBase salary, 3 published sources

A typical base salary range for a mid to senior hire, read from published adverts and salary surveys as at November 2026. Pay for new roles moves fast and varies with stage, location and scope, so we benchmark it for your role when we start a search.

Where these figures come from

Interview questions for a Reward Modelling Researcher

  1. Tell me about a reward model you trained. How did you decide it was good enough to hand to the team running reinforcement learning?

    Strong answers cover held-out agreement, comparison against simple baselines and checks beyond headline accuracy. They name what was still weak. Worry if the only evidence is accuracy on the training distribution, or if nothing was tested downstream.

  2. A policy's reward keeps rising but outputs look worse to reviewers. What do you do?

    Look for a diagnosis of over-optimisation: inspect high-reward samples, check for length or formatting shortcuts, compare against independent comparisons. Fixes might include regularisation, retraining on new samples or ensembles. Worry if they simply blame the policy.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are hiring a Reward Modelling Researcher to build the evaluative signals our post-training work depends on. We train models to learn from preference comparisons, then test hard how far those models can be trusted. The work sits between research and engineering, and shapes how our models are tuned.

What you will do

  • Train and iterate on reward models from preference comparisons
  • Design experiments that measure agreement on independent comparisons
  • Analyse reward models for bias and exploitability
  • Study the effect of preference noise and disagreement on the signal
  • Work with reinforcement learning colleagues to detect and fix reward gaming
  • Document methods and findings so others can reproduce them

What you will bring

  • Demonstrated work training and evaluating reward or preference models
  • Hands-on experience fine-tuning deep learning models
  • Sound experiment design, with clean held-out evaluation
  • Working knowledge of reinforcement learning and how policies exploit rewards
  • Fluency in a deep learning framework and experiment tracking
  • Clear written analysis that others can act on

Nice to have

  • Experience with robustness or out-of-distribution evaluation
  • Published or shared work on preference learning
  • Experience building evaluation sets for language models

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Reinforcement learningAdvancedOptimise policies from interaction and reward
  • Preference learningAdvancedLearn behaviour from comparisons and reward signals
  • Deep learningWorkingBuild and diagnose trainable neural models
  • Model evaluationWorkingMeasure capability and failure modes credibly

Other roles in post-training and preference optimisation

Hiring nowPost-Training Research EngineerA Post-Training Research Engineer adapts pretrained models into ones that behave as the company needs. They turn preference data and reward signals…Hiring nowReinforcement Learning Research EngineerA Reinforcement Learning Research Engineer improves how a model behaves by training it against interactive signals. They matter because reward gains…

Related roles in other families

EmergingReasoning Research ScientistReasoning, planning and searchA Reasoning Research Scientist tests and builds methods that make a model's multi-step inference reliable. They show the company which approaches…Hiring nowPlanning Algorithms ResearcherReasoning, planning and searchA Planning Algorithms Researcher builds planners that choose sequences of actions under hard constraints. Companies hire them so agents and systems…EmergingInference-Time Compute ResearcherReasoning, planning and searchAn Inference-Time Compute Researcher decides how much thinking a model should spend on each problem. They raise accuracy on hard tasks while keeping…EmergingNeuro-Symbolic AI ResearcherReasoning, planning and searchA Neuro-Symbolic AI Researcher combines learned models with symbolic rules and solvers, so systems reason consistently where rules matter. That…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Post-training and preference optimisationAll new job families

Start a search

Hiring a Reward Modelling Researcher?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.