Hiring nowMultimodal and generative world models

Multimodal Research Scientist

Also called Vision-Language Researcher

A Multimodal Research Scientist builds models that connect text, images, sound and action, and designs the benchmarks that show where they break. Companies need them to ship products that stay coherent across every input and output.

What they do

Trains and adapts models that take in and produce more than one modality, such as text with images, audio or action sequences. Designs the data mixtures, objectives and architectures that tie those modalities together. Builds cross-modal benchmarks that expose mismatches, for example a caption that contradicts the image or speech that drifts from the script. Runs ablations, reads failure cases by hand and decides what to change next. Works with simulators and media evaluators where real data is thin. Writes up results so engineers and product teams know what the model can and cannot do.

What a strong hire delivers

Within the first months, a strong hire delivers a working multimodal model that beats the current baseline on the cases the business cares about. They also leave a cross-modal benchmark the team trusts and reruns. Failure rates are measured by modality combination, not as one blended score. The team ends up with a clear view of which gaps are data problems, which are modelling problems, and what to try next.

How you can tell they are good

Ask them to walk through a multimodal model they built and where it failed. A strong candidate names specific failures, such as one modality dominating the others, and explains how they measured and fixed them. Ask how they would test a modality pair they have never worked with. Listen for evaluation design, scepticism about headline scores, and honest accounts of what did not work.

Why it is hard to fill

Titles are inconsistent. The same person may be called a vision-language researcher, a generative modelling scientist or a simulation researcher. Most candidates are deep in one modality and thin in the others. Few have owned evaluation as well as training. The strongest are often held tightly by frontier labs and research groups, so search by the work they have published or shipped, not by job title.

Where to find them

Look in multimodal ML teams, generative media groups and simulation or robotics research labs. Common titles include vision-language researcher, research scientist in speech and audio, and applied scientist in generative models. Many come from frontier labs, university research groups and well-funded startups building media or embodied products. Open-source contributors with released multimodal work are also worth a look.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Multimodal Research Scientist

  1. Describe a multimodal model you trained. Which modality combinations failed most, and how did you find out?

    Strong answers name concrete failure patterns, show per-combination measurement and explain the fix. Worry if they quote one aggregate score, blame data vaguely or cannot say what they would do differently.

  2. How would you build a benchmark for consistency between generated images and text?

    Look for clear definitions of consistency, varied and adversarial cases, human spot checks alongside automated evaluators, and awareness of leakage. Worry if the benchmark only measures what the model already does well.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are building products that work across text, images, sound and action, and we need a researcher to make our models coherent across all of them. You will train multimodal models, design the benchmarks that test them and help decide what ships.

What you will do

  • Train and adapt multimodal models across input and output modalities
  • Design data mixtures, objectives and architectures for cross-modal alignment
  • Build and maintain cross-modal benchmarks and evaluators
  • Analyse failures by modality combination and propose fixes
  • Use simulators to extend coverage where real data is limited
  • Share results and limits clearly with engineering and product teams

What you will bring

  • Demonstrated work training multimodal or vision-language models
  • Strong grounding in transformer modelling
  • Experience designing evaluations, not only running them
  • Ability to diagnose failures through ablations and error analysis
  • Comfort with multimodal training stacks at realistic scale
  • Clear written and spoken communication of results

Nice to have

  • Experience with generative media models
  • Work with simulation or digital twins
  • Published or open-source multimodal research

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Multimodal modellingAdvancedCombine information across modalities
  • Generative mediaAdvancedProduce controllable visual, sound or spatial outputs
  • Transformer modellingWorkingImplement attention-based sequence or multimodal models
  • Simulation and digital twinsWorkingBuild useful virtual representations of environments

Other roles in multimodal and generative world models

EmergingGenerative Video ResearcherA Generative Video Researcher builds models that produce video that stays consistent over time and responds to control. They make video generation…EmergingWorld Model ResearcherA World Model Researcher builds models that predict how an environment will change, given what has happened and what an agent does next. That gives…EmergingGenerative 3D ResearcherA Generative 3D Researcher builds models and pipelines that produce spatial assets with controllable structure. They matter because generated…

Related roles in other families

Hiring nowPretraining Research EngineerFoundation model pretrainingA pretraining research engineer runs controlled experiments on how large models learn from broad data. The work decides which architectures, data…Hiring nowModel Architecture ResearcherFoundation model pretrainingA Model Architecture Researcher tests changes to how a model is built, to improve its behaviour or efficiency. The work tells a company which…EmergingScaling Science ResearcherFoundation model pretrainingA Scaling Science Researcher works out how model performance changes with compute, data and model size. Their estimates tell a company how to spend…EmergingTraining Dynamics ResearcherFoundation model pretrainingA Training Dynamics Researcher works out why large training runs wobble, stall or diverge, and fixes the cause. Fewer failed runs means less wasted…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Multimodal and generative world modelsAll new job families

Start a search

Hiring a Multimodal Research Scientist?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.