Hiring nowForecasting, optimisation and recommendation

Ranking and Search Scientist

Also called Search Relevance Scientist

A Ranking and Search Scientist improves the order in which items appear for a defined information need. Better ordering means users find what they want sooner, and the product feels more useful and trustworthy.

What they do

They take a search or recommendation surface and work out what a good ordering looks like. They build a judged relevance set, then train and test ranking models against it. Most of the week goes on error analysis: which queries fail, why, and what signal is missing. They run offline evaluations and design online experiments. They watch latency, because a better model that is too slow does not ship. They also check how user clicks feed back into training, and whether that skews results over time. They work with engineers on serving and with product on what relevance means.

What a strong hire delivers

In the first months, a strong hire delivers a judged relevance set the team trusts and a baseline ranking model with clear evaluation. They show a measured improvement on relevance within an agreed latency budget. They document failure cases and set out how feedback loops could bias results, with a plan to limit that. The team ends up with a repeatable way to test ranking changes.

How you can tell they are good

Ask them to walk through a ranking system they improved, including how they judged relevance and what they measured. Strong people explain why the metric fits the information need, not just the score. Ask what went wrong after launch. They should talk readily about bias in click data, latency trade-offs and cases where the simpler model won. Be wary of answers that only name algorithms.

Why it is hard to fill

The title varies widely. The same work is called search relevance, ranking, recommendation or applied science. Many candidates come from data science, operations research or applied mathematics and have only touched ranking. Few have built a judged relevance set and run learning to rank through to production. People with real depth are often tied to a single large search product and are not actively looking.

Where to find them

Look in search and discovery teams at marketplaces, e-commerce, media, travel and job platforms. Titles include search relevance scientist, applied scientist, data scientist on ranking, and recommendation engineer. Operations research and applied mathematics backgrounds are a good source, especially for people who moved into information retrieval. Teams building retrieval for AI products are also growing a pool of relevant people.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Ranking and Search Scientist

  1. Tell me about a ranking model you improved. What was the information need and how did you know it got better?

    A strong answer defines the need clearly, names the metrics used and links them to user outcomes. They explain the baseline and the evidence of improvement. Worry if they only quote a headline score or cannot say what users gained.

  2. How would you build a judged relevance set for a new search product with no history?

    Good answers cover sampling queries, writing clear judging guidelines, checking agreement between judges and covering hard and rare queries. They mention keeping the set fresh. Worry if they would simply use clicks as truth.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are looking for a Ranking and Search Scientist to improve how our product orders results for the people using it. You will build ranking models and the judged relevance sets that measure them, and work closely with engineering and product to ship changes that users notice.

What you will do

  • Define relevance for our key search and recommendation surfaces with product colleagues
  • Build and maintain judged relevance sets with clear guidelines
  • Train and evaluate ranking models, including learning to rank approaches
  • Analyse failures by query type and propose fixes
  • Design and run online experiments to confirm gains
  • Track latency and keep models within agreed limits
  • Monitor and reduce the effect of feedback loops on results

What you will bring

  • Shipped a ranking or relevance model that improved a measured outcome
  • Built or used a judged relevance set and can explain its limits
  • Strong grounding in probability, statistics and experimental design
  • Practical experience with learning to rank
  • Comfortable writing production-quality Python and using scikit-learn
  • Can explain trade-offs between relevance and latency to non-specialists

Nice to have

  • Background in operations research or optimisation
  • Experience with time-series or forecasting methods
  • Experience with counterfactual evaluation of ranking

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Time-series modellingAdvancedModel temporal dependencies and forecasts
  • Optimisation and operations researchAdvancedChoose actions subject to objectives and constraints
  • Probability and statisticsWorkingQuantify variability and statistical evidence
  • Experimental designWorkingChoose comparisons that identify credible effects

Other roles in forecasting, optimisation and recommendation

Hiring nowForecasting Data ScientistA Forecasting Data Scientist produces calibrated demand and operational forecasts that planning, inventory and staffing decisions can rely on…Hiring nowOperations Research AnalystAn Operations Research Analyst turns messy business decisions into optimisation models with explicit constraints. They help a company allocate…Hiring nowRecommendation Systems EngineerA Recommendation Systems Engineer builds the ranking systems that decide what each user sees, and keeps them honest with feedback controls. Good…

Related roles in other families

Hiring nowQuantitative ML ResearcherInvestment, quantitative finance and market AIA Quantitative ML Researcher tests investment ideas with models and builds evidence that holds up after costs and constraints. They stop the company…Hiring nowAI Portfolio Analytics ScientistInvestment, quantitative finance and market AIAn AI Portfolio Analytics Scientist models portfolio risk and the tradeoffs behind investment decisions. They give the company reliable risk…Hiring nowAI Execution ResearcherInvestment, quantitative finance and market AIAn AI Execution Researcher builds and tests models that decide how to trade orders under market and cost constraints. Done well, it lowers the cost…EmergingInvestment Research AI Workflow SpecialistInvestment, quantitative finance and market AIThis person builds AI assistance for investment research where every claim links back to its source and an analyst can check it. It matters because…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Forecasting, optimisation and recommendationAll new job families

Start a search

Hiring a Ranking and Search Scientist?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.