Hiring nowInference and model optimisation

Edge AI Engineer

Also called Embedded ML Engineer · TinyML Engineer

An Edge AI Engineer gets learned models running on constrained devices and offline environments, within clear limits on power, memory and latency. It matters because a model that only works in the cloud cannot ship in the product.

What they do

They take a model built elsewhere and make it run on the target device. That starts with profiling to find where time, memory and energy go. They then apply quantisation, pruning, distillation or architecture changes, and test what each one costs in task quality. They write or tune kernels for the specific chip, manage memory layout and handle runtime limits. They run the model on real hardware, not only emulators, and test it in field conditions such as heat, poor connectivity and long uptime. They record results so the team can see the trade-offs.

What a strong hire delivers

In the first months, a strong hire gets one real model running on real target devices and shows the numbers for power, memory, latency and task quality. They leave behind a repeatable benchmarking setup on actual hardware. They produce a field validation report that names where the model fails and what was traded away. The team then knows what is shippable and what is not.

How you can tell they are good

Ask them to walk through a model they shipped on a device. Strong people give measured numbers, name the bottleneck they found first, and explain which optimisation hurt quality and how they caught it. They talk about hardware limits without prompting. Be wary of anyone who only reports benchmark results from a workstation or a simulator, or who treats compression as a single setting.

Why it is hard to fill

The title varies widely: embedded ML, TinyML, on-device inference, performance engineer. Few people sit across both ML and low-level hardware work. Embedded engineers often lack model knowledge, and ML engineers often have never touched a constrained device. Most good candidates are in post and learned the mix by necessity, so job titles say little. Judge them on shipped work.

Where to find them

Look in performance engineering, ML systems and distributed services teams, where people already care about latency and memory. Embedded software engineers who moved into ML are a strong source. So are compiler and runtime engineers, and GPU programmers. Typical employers make devices, robots, vehicles, wearables or industrial equipment, plus AI companies pushing models to the edge and into offline settings.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an Edge AI Engineer

  1. Describe a model you got running on a constrained device. What were the limits and what did you change?

    A strong answer names the device, the limits and the measured starting point. It explains each change and its effect on quality. Worry if the story is vague, uses only emulators, or claims large gains without numbers.

  2. How do you decide where to start when a model is too slow or too large for the target hardware?

    They profile first and find the real bottleneck, whether compute, memory bandwidth or data movement. They pick the cheapest fix before the clever one. Worry if they jump straight to quantisation without measuring.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are hiring an Edge AI Engineer to get our models running on constrained devices and in offline environments. The work sits between machine learning and hardware. You will make models fit tight limits on power, memory and latency without losing the quality our users need.

What you will do

  • Profile models on target devices and find the real bottlenecks
  • Apply quantisation, pruning, distillation and runtime tuning, and measure the effect on quality
  • Write or tune kernels and inference code for specific hardware
  • Build repeatable benchmarks that run on real devices
  • Run field validation and write up results, failures and trade-offs
  • Agree quality and resource limits with product and research teams

What you will bring

  • Shipped at least one model on a constrained or embedded device, with measured results
  • Hands-on experience with model compression and quantisation
  • Strong systems skills in C++ or a similar language, and in profiling tools
  • Background in performance engineering, ML systems or distributed services
  • Able to evaluate task quality, not only headline accuracy
  • Clear written reporting of trade-offs

Nice to have

  • GPU or accelerator programming experience
  • Experience with offline or intermittently connected deployments
  • Familiarity with inference serving as well as on-device work

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Inference servingAdvancedServe models reliably under workload constraints
  • Model compressionAdvancedReduce model compute and storage requirements
  • GPU programmingWorkingOptimise parallel accelerator kernels
  • Distributed systemsWorkingDesign reliable work across networked machines

Other roles in inference and model optimisation

Hiring nowInference Systems EngineerAn Inference Systems Engineer builds the serving engines and runtime services that run models in production. They keep latency low and cost under…Hiring nowModel Compression EngineerA Model Compression Engineer makes trained models smaller and faster to run without letting quality slip past agreed limits. For a company, that…Hiring nowInference Performance EngineerAn Inference Performance Engineer profiles how a model is served, finds where time and accelerator capacity are lost, and fixes it without hurting…

Related roles in other families

Hiring nowDistributed Training EngineerDistributed training and research systemsA Distributed Training Engineer scales model training across many accelerators while keeping results correct. They keep expensive training runs…Hiring nowResearch Infrastructure EngineerDistributed training and research systemsA research infrastructure engineer gives researchers dependable compute and software environments, so experiments run, repeat and finish without…EmergingTraining Reliability EngineerDistributed training and research systemsA Training Reliability Engineer keeps long training runs alive and trustworthy. They detect failures, recover quickly and prove the recovered run is…Hiring nowResearch Productivity EngineerDistributed training and research systemsA Research Productivity Engineer builds the tools and automation that let researchers run experiments, analyse results and reproduce them faster. It…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Inference and model optimisationAll new job families

Start a search

Hiring an Edge AI Engineer?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.