Hiring nowInference and model optimisation

Inference Performance Engineer

Also called Performance Engineer, Inference Systems

An Inference Performance Engineer profiles how a model is served, finds where time and accelerator capacity are lost, and fixes it without hurting quality. It cuts latency and serving cost for the product.

What they do

They take a real workload and measure it from request to response. They profile at the kernel level and at the service level, then find where the time goes. That might be scheduling, batching, memory movement, a slow kernel or a poorly sized cache. They try fixes such as quantisation, better batching, kernel changes or changes to how requests are routed. After each change they check output quality against agreed limits. They write up what they found, what they changed and what it did. They work closely with model, infrastructure and product teams.

What a strong hire delivers

In the first months, a strong hire delivers a clear performance baseline on representative workloads and a ranked list of bottlenecks. They ship a few verified optimisations that lower latency and raise accelerator utilisation, each with evidence that quality held. They also leave behind repeatable benchmarks and profiling habits, so the team keeps finding gains after they have moved on.

How you can tell they are good

Ask them to walk through a past optimisation from first measurement to verified result. A strong candidate names the workload, the profiling method, what they ruled out and how they checked quality. They explain trade-offs plainly. Be wary of anyone who quotes headline speed-ups from toy benchmarks, or who cannot say what quality limit they worked to.

Why it is hard to fill

Titles vary a great deal, and few people have worked across both kernels and live services. Most candidates are strong on one side. Performance engineers from other systems often lack GPU and model knowledge. ML engineers often lack profiling discipline. Demand is high, and good people tend to be settled where they are. We usually have to look at adjacent backgrounds and test for depth.

Where to find them

They come from performance engineering, ML systems, GPU programming and distributed services teams. Look at inference teams at model providers and frontier labs, at companies that host models for others, and at infrastructure groups running large serving fleets. Some come from high-performance computing, graphics, or database and compiler work. Open-source serving projects are another good place to spot real contributors.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an Inference Performance Engineer

  1. Tell me about an inference bottleneck you found that was not where the team expected it. How did you locate it?

    A strong answer describes a specific workload, the tools and measurements used, and how they ruled out the obvious suspects. Worry if the story is only about a known fix applied, with no real diagnosis.

  2. How do you decide a workload is representative enough to optimise against?

    Good answers cover real request mixes, input and output length distributions, concurrency and tail latency. They mention checking with production traces. Be cautious if they rely on a single average or a synthetic benchmark.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We serve learned models to real users, and speed and cost matter to us. We are looking for an Inference Performance Engineer to profile our serving stack, remove bottlenecks and make sure every gain holds up against our quality limits. You will work with model and infrastructure engineers.

What you will do

  • Profile inference workloads from request to response at kernel and service level
  • Find and rank bottlenecks in latency and accelerator utilisation
  • Design and test optimisations such as quantisation, batching and kernel changes
  • Verify that each change keeps output quality within agreed limits
  • Build repeatable benchmarks on representative workloads
  • Write clear performance analyses for engineering and product teams

What you will bring

  • Shown work cutting latency or cost in a live serving system
  • Hands-on kernel and service profiling experience
  • Working knowledge of GPU programming
  • Experience with distributed systems
  • Familiarity with inference serving frameworks and model compression
  • Clear written analysis that others can act on

Nice to have

  • Contributions to open-source serving or kernel projects
  • Experience with custom kernel development
  • Background in high-performance computing

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Inference servingAdvancedServe models reliably under workload constraints
  • Model compressionAdvancedReduce model compute and storage requirements
  • GPU programmingWorkingOptimise parallel accelerator kernels
  • Distributed systemsWorkingDesign reliable work across networked machines

Other roles in inference and model optimisation

Hiring nowInference Systems EngineerAn Inference Systems Engineer builds the serving engines and runtime services that run models in production. They keep latency low and cost under…Hiring nowModel Compression EngineerA Model Compression Engineer makes trained models smaller and faster to run without letting quality slip past agreed limits. For a company, that…Hiring nowEdge AI EngineerAn Edge AI Engineer gets learned models running on constrained devices and offline environments, within clear limits on power, memory and latency…

Related roles in other families

Hiring nowDistributed Training EngineerDistributed training and research systemsA Distributed Training Engineer scales model training across many accelerators while keeping results correct. They keep expensive training runs…Hiring nowResearch Infrastructure EngineerDistributed training and research systemsA research infrastructure engineer gives researchers dependable compute and software environments, so experiments run, repeat and finish without…EmergingTraining Reliability EngineerDistributed training and research systemsA Training Reliability Engineer keeps long training runs alive and trustworthy. They detect failures, recover quickly and prove the recovered run is…Hiring nowResearch Productivity EngineerDistributed training and research systemsA Research Productivity Engineer builds the tools and automation that let researchers run experiments, analyse results and reproduce them faster. It…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Inference and model optimisationAll new job families

Start a search

Hiring an Inference Performance Engineer?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.