Model Architecture Researcher

Also called Neural Architecture Researcher

A Model Architecture Researcher tests changes to how a model is built, to improve its behaviour or efficiency. The work tells a company which architectural bets are worth the compute before they commit to a large training run.

What they do

They propose architectural changes, implement them in code and run controlled comparisons against a baseline. The changes might touch attention variants, state-space layers or hybrids of the two. They plan each experiment so that data, compute and training setup are held steady, then read the results with care. Much of the day is spent writing and debugging training code, checking runs for faults, and working out whether a gain is real or noise. They write up findings for the wider pretraining team and say plainly when an idea failed. They also work with data and infrastructure colleagues, since architecture results depend on both.

What a strong hire delivers

In the first months, a strong hire delivers a clean, reproducible experiment setup that others can reuse. They produce working implementations of one or two candidate architectures and a controlled comparison against the current baseline. They give the team a clear recommendation on what to scale, what to drop and why. Along the way they tighten how the team judges evidence, so decisions rest on fair comparisons.

How you can tell they are good

Ask them to walk through an architecture comparison they ran. Listen for how they held data and compute constant, how they chose baselines and how they handled run-to-run variance. Ask what they would do if a promising result vanished at larger scale. A strong candidate talks about failed ideas openly, shows code they wrote themselves and can explain why a change should help, not just that it did.

Why it is hard to fill

Few people have run real architecture experiments with the discipline this work needs. Many candidates have used standard models or tuned them, but have not designed fair comparisons. Titles vary widely, and the same work may sit under research, pretraining or modelling. Strong people are often held in place by interesting problems and access to compute. Public output can be thin, so judging depth takes careful conversation.

Where to find them

They tend to come from deep-learning research teams, frontier labs, and academic groups that train their own models. Look at people titled research scientist, research engineer or machine learning researcher, especially those who have worked on sequence modelling, efficient attention or state-space methods. Some come from applied teams at model providers who moved toward core modelling. Open-source model contributors can also be a good source.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Model Architecture Researcher

Describe an architectural change you tested. How did you design the comparison?
A strong answer names the baseline, what was held constant (data, compute, tokens, seeds) and how they judged significance. Worry if they only report a final metric, ignore variance or cannot say what else changed between runs.
A hybrid of attention and state-space layers beats your baseline at small scale. What do you do next?
Good answers check for confounds, repeat with seeds, plan scaling tests and consider throughput as well as quality. Worry about anyone who would scale straight away or treat small-scale wins as settled.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are hiring a Model Architecture Researcher to test changes to how our models are built. You will design and run controlled experiments that show which ideas improve behaviour or efficiency, and help decide what we scale in pretraining.

What you will do

Propose and implement architectural changes, including attention, state-space and hybrid designs
Design controlled comparisons that hold data and compute constant
Run and monitor training experiments and diagnose faults
Analyse results and separate real gains from noise
Write clear reports with recommendations for the pretraining team
Work with data and infrastructure colleagues to keep experiments fair and reproducible

What you will bring

Demonstrated deep-learning research with models you trained yourself
Strong experimental design and sound statistical judgement
Fluent in PyTorch or JAX, and familiar with Transformer implementations
Evidence of reproducible training work others could rerun
Understanding of how data quality and provenance affect results
Clear written communication of results, including negative ones

Nice to have

Experience with state-space or hybrid sequence models
Work on efficient training or inference
Open-source or published modelling work

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

Deep learningAdvancedBuild and diagnose trainable neural models
Transformer modellingAdvancedImplement attention-based sequence or multimodal models
Experimental designWorkingChoose comparisons that identify credible effects
Data quality and provenanceWorkingTrack dataset origin, validity and transformations

Other roles in foundation model pretraining

Hiring nowPretraining Research EngineerA pretraining research engineer runs controlled experiments on how large models learn from broad data. The work decides which architectures, data…EmergingScaling Science ResearcherA Scaling Science Researcher works out how model performance changes with compute, data and model size. Their estimates tell a company how to spend…EmergingTraining Dynamics ResearcherA Training Dynamics Researcher works out why large training runs wobble, stall or diverge, and fixes the cause. Fewer failed runs means less wasted…

Related roles in other families

Hiring nowMachine Learning Research ScientistLearning theory and statistical machine learningA Machine Learning Research Scientist develops and tests learning methods against explicit hypotheses. They give the company evidence about what…Hiring nowStatistical Learning ScientistLearning theory and statistical machine learningA Statistical Learning Scientist builds prediction methods that show their assumptions and their uncertainty. For a company shipping models into…Hiring nowLearning Theory ResearcherLearning theory and statistical machine learningA Learning Theory Researcher works out what learning algorithms can and cannot do, and proves it. For an AI company, that means firmer ground for…Hiring nowAutomated Machine Learning ResearcherLearning theory and statistical machine learningAn Automated Machine Learning Researcher builds search procedures that choose and tune models within a fixed compute budget. It matters because…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

Careers: jobs, Anthropic
Transformers documentation, Hugging Face

← Foundation model pretraining All new job families

Start a search

Hiring a Model Architecture Researcher?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.