Hiring nowAI observability, reliability and FinOps

AI Site Reliability Engineer

Also called AI SRE · ML Reliability Engineer

An AI Site Reliability Engineer keeps services that depend on models available and recoverable under real traffic. They matter because customers judge an AI product by whether it works when they need it.

What they do

They set service objectives for products that rely on model calls, retrieval and agent steps. They add tracing and telemetry so a slow or failing request can be followed through every dependency. They run load tests to find where latency, rate limits or queueing break the service. They write runbooks for the failures that actually occur, such as provider outages, degraded output quality and runaway retries. They run incident drills, lead or support live incidents, and write the reviews afterwards. They also watch cost dashboards, because reliability choices such as fallbacks and retries change spend.

What a strong hire delivers

In the first months, a strong hire agrees service objectives with product and engineering that people actually use. Tracing covers the main request paths, so the team can see where failures start. Runbooks exist for the most likely incidents, and at least one drill has been run and learned from. Recovery gets faster and failure-budget use becomes visible. The team also has a clearer view of what reliability costs.

How you can tell they are good

Ask them to walk through an incident they handled end to end. Listen for how they found the cause, what they changed afterwards and how they measured it. Ask how they would define an objective for a service whose output can be wrong without any error. Strong people separate availability from quality, name trade-offs with cost, and talk about drills, not just dashboards.

Why it is hard to fill

Titles are inconsistent. The work appears as AI SRE, ML reliability, platform or infrastructure, and many job ads ask for everything. Most good SREs have not yet worked with model dependencies, and most ML engineers have not carried a pager. The people who have done both are few. Candidates from FinOps or performance backgrounds can be strong but are easy to overlook on a CV.

Where to find them

Look first at site reliability, performance engineering and cloud operations teams, especially where services call external model providers or run their own inference. Platform and MLOps engineers who have joined on-call rotations are another source. Some come from FinOps and cost engineering. Product-led AI companies, infrastructure-heavy start-ups and larger firms with AI platform teams are the likeliest places to find them.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an AI Site Reliability Engineer

Tell me about an incident in a service with an external dependency. What did you do during it and what changed afterwards?
A strong answer gives a clear timeline, the diagnosis route, the immediate fix and the lasting change, with a measure of improvement. Worry if they blame the dependency, describe only heroics, or cannot say what was learned.
How would you set service objectives for a feature where the model can return a confident wrong answer with no error?
They should separate availability and latency from output quality, and suggest sampled evaluation or user signals as indicators. Good answers admit imperfect measures. Worry if they only mention uptime or want to measure everything.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We build products that depend on models, and our customers expect them to work every time. We are looking for an AI Site Reliability Engineer to set reliability standards for those services, find weak points before users do, and help the team recover quickly when things fail.

What you will do

Define and maintain service objectives for model-dependent services
Build tracing and telemetry across model calls, retrieval and agent steps
Run load tests and act on what they show
Write and maintain runbooks for likely failures
Plan and run incident drills, and take part in live incident response
Track failure-budget use and report on recovery time
Work with engineering and finance on the cost effects of reliability choices

What you will bring

Demonstrated work running production services with on-call responsibility
Experience building tracing, telemetry and alerting that teams relied on
Examples of leading or contributing to incident reviews that led to change
Familiarity with services that depend on external or hosted models
Ability to explain reliability trade-offs to non-specialists
Practical experience with load testing

Nice to have

Background in FinOps or cost analysis for cloud workloads
Experience with model evaluation or the model lifecycle
Experience with agent-based systems in production

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

Reliability and incident responseAdvancedDetect failures, restore service and prevent recurrence
MLOps and model lifecycleAdvancedVersion, release, monitor and retire models
Commercial and financial analysisWorkingAssess unit economics and value realisation
Model evaluationWorkingMeasure capability and failure modes credibly

Other roles in aI observability, reliability and FinOps

EmergingAI Observability EngineerAn AI Observability Engineer instruments model and agent workflows so teams can see what is happening in production. It matters because failures…Hiring nowAI FinOps AnalystAn AI FinOps Analyst measures what each useful AI task costs and finds ways to bring it down without hurting quality. They give the company a clear…Hiring nowAI Capacity Planning EngineerAn AI Capacity Planning Engineer plans how much compute the company needs, when, and at what cost. They keep services reliable at peak load without…EmergingAgent Reliability EngineerAn Agent Reliability Engineer keeps AI agents dependable in production. They set service targets, watch quality, cost and drift, and make sure…

Related roles in other families

Hiring nowMachine Learning EngineerApplied AI and machine learning engineeringA Machine Learning Engineer turns trained models into tested, dependable services that run inside real products. They make sure model quality holds…Hiring nowAI Application EngineerApplied AI and machine learning engineeringAn AI Application Engineer builds the user-facing software that joins a model to business logic. They turn model capability into a product that…Hiring nowApplied AI ScientistApplied AI and machine learning engineeringAn Applied AI Scientist takes a research method and makes it work on a real product problem. They prove it beats a simple baseline on representative…Hiring nowForward Deployed AI EngineerApplied AI and machine learning engineeringA Forward Deployed AI Engineer works inside a customer's environment to get AI solutions running on real tasks. They matter because a model only…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

AI Risk Management Framework, NIST
MLflow documentation, MLflow contributors
vLLM documentation, vLLM contributors

← AI observability, reliability and FinOps All new job families

Start a search

Hiring an AI Site Reliability Engineer?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.