Hiring nowComputer vision and spatial perception

Video Understanding Scientist

Also called Video AI Scientist

A Video Understanding Scientist builds models that recognise actions, events and the dependencies between them in video. Companies hire for this work so their products can reason about what happens over time, not just what appears in a single frame.

What they do

The work starts with a question about video: what happened, when, and because of what. The scientist turns that into a modelling problem, then designs the experiments to answer it. They build and train spatiotemporal models in PyTorch, using OpenCV for decoding, sampling and preprocessing. They decide how to represent time, how long a clip needs to be, and how to handle motion, occlusion and camera change. They build evaluation sets that split by time, so the model cannot lean on near-duplicate footage. They study failures clip by clip, then adjust the model or the data pipeline.

What a strong hire delivers

In the first months, a strong hire delivers a working video model with a clear baseline to beat. They also deliver a benchmark that is held out by time, so results mean something. Alongside it come error analysis the team can act on, honest uncertainty estimates, and a short list of what to try next, ranked by likely gain and cost.

How you can tell they are good

Ask them to walk through a video project end to end. Strong candidates explain how they split data to avoid leakage across time, why they chose their temporal window, and how they scored event timing as well as event class. They should show failure cases unprompted and say what they would not trust the model to do.

Why it is hard to fill

Few people have done real work on time in video, as opposed to image models applied frame by frame. Titles vary widely, and many candidates call themselves computer vision engineers while having never built a temporal benchmark. Others come from research with strong papers but little experience of shipping. We screen for evaluation judgement first, because it separates the two groups quickly.

Where to find them

Most come from computer vision, image processing or applied machine learning teams. Look at people working on action recognition, tracking, video retrieval or surveillance analytics. Some sit in robotics, sports analytics, media technology or autonomous systems companies. Others are in university labs or at a frontier lab, usually under titles such as research scientist or applied scientist.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Video Understanding Scientist

  1. Describe a video model you built. What did it need to understand about time that a frame-by-frame model could not?

    A strong answer names a specific temporal dependency, such as order of events or duration, and shows how the architecture captured it. Worry if they describe only image models run on frames, or cannot say what time added.

  2. How would you build a temporally held-out benchmark for a new video task?

    They split by time or source, not random clips, and explain the leakage risk from near-duplicate footage. They discuss class balance and label quality. Worry if they propose a random split or never mention leakage.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are hiring a Video Understanding Scientist to build models that recognise actions, events and the dependencies between them in video. The work runs from framing the problem to shipping a model and a benchmark the whole team can trust. You will work closely with engineering and product.

What you will do

  • Build and train spatiotemporal models for action and event understanding
  • Design temporally held-out benchmarks and evaluation protocols
  • Measure event accuracy and temporal localisation quality
  • Run experiments and report results with clear uncertainty
  • Analyse failure cases and turn them into model or data improvements
  • Build video preprocessing and sampling pipelines
  • Explain findings and trade-offs to non-specialist colleagues

What you will bring

  • Demonstrated work on video models, such as action recognition, detection or localisation
  • Strong deep learning skills in PyTorch
  • Experience with OpenCV or similar for video processing
  • Evidence of careful experimental design and leakage-aware evaluation
  • Practical experience with spatiotemporal representation learning
  • Ability to quantify and communicate uncertainty

Nice to have

  • Experience with long-video or streaming settings
  • Published or open-source work on video benchmarks
  • Experience deploying models under compute limits

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Computer visionAdvancedAnalyse or generate visual information
  • Deep learningAdvancedBuild and diagnose trainable neural models
  • Experimental designWorkingChoose comparisons that identify credible effects
  • Uncertainty quantificationWorkingEstimate confidence, calibration and model uncertainty

Other roles in computer vision and spatial perception

Hiring nowComputer Vision EngineerA Computer Vision Engineer builds detection and recognition systems that work in a defined setting, and proves how well they work. Companies rely on…Hiring now3D Vision ResearcherA 3D Vision Researcher recovers spatial structure from images and depth data, then proves it holds up. Companies building robotics, mapping, AR or…EmergingOpen-World Perception ResearcherAn Open-World Perception Researcher builds vision models that know when they are looking at something unfamiliar. That matters because a system that…

Related roles in other families

Hiring nowScientific Machine Learning ScientistScientific machine learning and research computingA Scientific Machine Learning Scientist builds learned models that obey the physics, chemistry or biology of a problem. They matter because a model…Hiring nowAI Research Software EngineerScientific machine learning and research computingAn AI Research Software Engineer turns scientific machine learning ideas into software that other people can run, trust and extend. They give a…EmergingAI Scientific Workflow EngineerScientific machine learning and research computingAn AI Scientific Workflow Engineer connects data, models and instruments into reproducible science pipelines. They make sure results can be traced…EmergingAI Reproducibility SpecialistScientific machine learning and research computingAn AI Reproducibility Specialist independently reruns AI-supported scientific claims and reports what holds and what does not. They protect the…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Computer vision and spatial perceptionAll new job families

Start a search

Hiring a Video Understanding Scientist?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.