Hiring nowInference and model optimisation

Model Compression Engineer

Also called Quantisation Engineer · Model Distillation & Efficiency Engineer · Inference Optimisation Researcher

A Model Compression Engineer makes trained models smaller and faster to run without letting quality slip past agreed limits. For a company, that means lower serving cost, lower latency and models that fit on the hardware customers actually have.

What they do

Take a trained model and make it cheaper to run. That usually means choosing a quantisation scheme, pruning weights or distilling a large model into a smaller one. Profile where time and memory go, then target those costs. Build evaluation sets that catch regressions the headline benchmark misses. Compare each compressed checkpoint with the original on quality, latency and memory. Work with inference serving and kernel engineers so the gains show up in production, not only in a notebook. Document what was traded away and why, so the team can approve the result.

What a strong hire delivers

In the first months, a strong hire ships a compressed checkpoint running in the serving stack, with a clear comparison against the baseline. Size and speed improve within error bounds the team has agreed. They also leave behind a repeatable evaluation harness and a written record of which methods worked, which failed and where quality dropped, so later compression work starts further along.

How you can tell they are good

Ask them to walk through a model they compressed. Strong candidates give the baseline, the method, the measured speed and size change, and the quality cost, in specifics. They explain why they chose one approach over another and how they found regressions on rare inputs. Ask what they would do if the gains vanished under real traffic. Be wary of answers that quote only average benchmark scores.

Why it is hard to fill

Very few people have done this end to end, and the title varies: quantisation engineer, distillation engineer, inference optimisation researcher. The work sits between research and systems, so most candidates lean one way. Those with real production results are usually well placed already, and frontier labs, edge AI companies and chip makers all compete for them. Searching by title alone misses much of the pool.

Where to find them

Look at performance engineers, ML systems engineers and people from distributed services who moved towards model efficiency. Kernel and GPU programmers are a strong source, as are researchers who published on quantisation or distillation. They tend to sit in frontier labs, edge AI companies, chip and accelerator makers, and inference platform teams. Open source contributors to inference engines are worth a close look.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for a Model Compression Engineer

  1. Tell me about a model you compressed. What was the baseline, the method, and the result on size, speed and quality?

    A strong answer gives concrete before and after numbers for all three, names the method and explains why it was chosen. Worry if they only describe the technique, cannot state the quality cost, or take credit for team work without detail.

  2. How would you decide between post-training quantisation, quantisation-aware training and distillation for a new model?

    Good answers weigh available compute, data access, target hardware, tolerance for quality loss and timeline. They mention trying the cheapest option first. Worry about a single favourite method applied to every case.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We build products that depend on serving large models quickly and affordably. We are hiring a Model Compression Engineer to make our models smaller and faster while holding quality within limits we agree together. You will work closely with inference, research and product colleagues.

What you will do

  • Compress models using quantisation, pruning and distillation
  • Profile models to find memory and compute bottlenecks
  • Build evaluation sets that catch quality regressions
  • Produce compressed checkpoints with clear quality and latency comparisons
  • Work with serving engineers to get gains into production
  • Document trade-offs so changes can be reviewed and approved

What you will bring

  • Shipped compressed models with measured size, speed and quality results
  • Hands-on experience with quantisation, pruning or distillation
  • Comfortable profiling GPU workloads and reading performance data
  • Experience with inference serving or distributed systems
  • Clear written comparisons of options and trade-offs
  • Sound judgement about when an evaluation is not enough

Nice to have

  • Custom kernel or GPU programming experience
  • Published or open source work on model efficiency
  • Experience targeting edge or specialised accelerators

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Inference servingAdvancedServe models reliably under workload constraints
  • Model compressionAdvancedReduce model compute and storage requirements
  • GPU programmingWorkingOptimise parallel accelerator kernels
  • Distributed systemsWorkingDesign reliable work across networked machines

Other roles in inference and model optimisation

Hiring nowInference Systems EngineerAn Inference Systems Engineer builds the serving engines and runtime services that run models in production. They keep latency low and cost under…Hiring nowInference Performance EngineerAn Inference Performance Engineer profiles how a model is served, finds where time and accelerator capacity are lost, and fixes it without hurting…Hiring nowEdge AI EngineerAn Edge AI Engineer gets learned models running on constrained devices and offline environments, within clear limits on power, memory and latency…

Related roles in other families

Hiring nowDistributed Training EngineerDistributed training and research systemsA Distributed Training Engineer scales model training across many accelerators while keeping results correct. They keep expensive training runs…Hiring nowResearch Infrastructure EngineerDistributed training and research systemsA research infrastructure engineer gives researchers dependable compute and software environments, so experiments run, repeat and finish without…EmergingTraining Reliability EngineerDistributed training and research systemsA Training Reliability Engineer keeps long training runs alive and trustworthy. They detect failures, recover quickly and prove the recovered run is…Hiring nowResearch Productivity EngineerDistributed training and research systemsA Research Productivity Engineer builds the tools and automation that let researchers run experiments, analyse results and reproduce them faster. It…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Inference and model optimisationAll new job families

Start a search

Hiring a Model Compression Engineer?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.