Hiring nowAI accelerators and hardware co-design

AI Accelerator Architect

Also called ML Hardware Architect · AI Chip / Accelerator Architect · Silicon Architect (AI)

An AI Accelerator Architect decides how a chip or compute system should be built for specific training or inference workloads. Their choices set the performance, energy use and silicon cost of everything the company runs on it.

What they do

They start with the workloads: which models, which operators, what batch sizes, where the time and energy actually go. They profile real code and build a picture of the bottlenecks in compute, memory and interconnect. Then they propose hardware structures, such as compute array shapes, memory hierarchy, on-chip networks and numeric formats. They model options in simulators, test them against workload traces, and write up the tradeoffs. They work daily with RTL designers, compiler and kernel engineers, and physical design teams, and they defend or revise the architecture as the evidence changes.

What a strong hire delivers

In the first months, a strong hire produces a workload study that the whole team accepts as the shared view of what the hardware must do. They follow it with an architecture specification that sets out the main structures, the options rejected and why. They also leave a working performance model that others can use to test changes, and a clear account of the performance, energy and area tradeoffs, checked against verified workloads.

How you can tell they are good

Ask them to walk through a past architecture decision from workload evidence to final design. Good candidates name the bottleneck, show the numbers they used, explain what they gave up and say what later proved wrong. Ask how they validated their simulator against real behaviour. Be wary of people who talk only in peak figures, cannot explain memory behaviour, or credit the whole design to a team without describing their own call.

Why it is hard to fill

Very few people have shaped an accelerator that reached silicon, and most of them are settled inside a handful of chip companies and labs. Titles vary widely, so searching by job title misses many. The skill sits between computer architecture, RTL and machine learning workloads, and candidates are usually strong in two of the three. Hiring managers often ask for all three, which narrows an already tiny pool.

Where to find them

Look at computer architecture teams at chip companies, especially those who worked on GPUs, neural processing units or custom accelerators. Labs building their own silicon employ performance modelling engineers and hardware-aware ML engineers who can step up. Others come from electrical engineering and chip design, or from academic architecture groups with strong simulation work. Compiler and kernel engineers who have shaped hardware decisions are a useful adjacent source.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an AI Accelerator Architect

  1. Describe an architecture decision you drove from workload analysis to a final design. What evidence changed your mind?

    A strong answer names the workload, the measured bottleneck and the options compared. It shows a real change of view driven by data. Worry if the story is vague, credits only the team, or rests on peak throughput claims.

  2. How would you decide the memory hierarchy for an inference chip aimed at large language model serving?

    Look for reasoning about bandwidth versus capacity, batch size, key-value cache behaviour and data reuse. They should ask about latency targets and cost. Worry if they treat compute as the only limit.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are building compute hardware around the AI workloads we run. We need an accelerator architect to decide how our chips should be structured, backed by evidence from real models. You will work closely with silicon, software and research teams.

What you will do

  • Analyse and profile target training and inference workloads
  • Propose compute, memory and interconnect structures for them
  • Build and maintain performance, energy and area models
  • Write architecture specifications and record rejected options
  • Work with RTL, compiler and kernel teams to refine the design
  • Check results against verified workloads and report tradeoffs

What you will bring

  • Shaped an accelerator or similar architecture and can explain their own decisions
  • Strong grounding in computer architecture and memory systems
  • Hands-on experience with simulators and workload profilers
  • Able to read and discuss RTL with design engineers
  • Solid understanding of machine learning workloads and their operators
  • Clear written communication of tradeoffs

Nice to have

  • Experience with a design that reached silicon
  • GPU programming experience
  • Familiarity with EDA flows and physical design constraints

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Hardware and computer architectureAdvancedDesign computing structures and hardware interfaces
  • GPU programmingAdvancedOptimise parallel accelerator kernels
  • Linear algebra and optimisationWorkingReason about matrix operations and objective functions
  • Simulation and digital twinsWorkingBuild useful virtual representations of environments

Other roles in aI accelerators and hardware co-design

Hiring nowAI Hardware Verification EngineerAn AI Hardware Verification Engineer proves that accelerator logic and its interfaces behave correctly before silicon is committed. Escaped defects…Hiring nowHardware-Software Co-Design EngineerA Hardware-Software Co-Design Engineer shapes model software and chip interfaces together, so neither is tuned in isolation. For an AI company, that…EmergingNeuromorphic Computing ResearcherA neuromorphic computing researcher tests whether event-driven, brain-inspired computation can beat conventional hardware on a real task. The answer…

Related roles in other families

Hiring nowAI Inference Compiler EngineerAI compilers, kernels and runtime systemsAn AI Inference Compiler Engineer makes trained models run faster and cheaper on real hardware. They change how model computation is compiled and…Hiring nowGPU Kernel EngineerAI compilers, kernels and runtime systemsA GPU Kernel Engineer writes and tunes the low-level code that runs model operations on accelerators. Faster, correct kernels cut compute cost and…Hiring nowML Runtime EngineerAI compilers, kernels and runtime systemsAn ML Runtime Engineer builds the scheduling, memory and execution layer that runs models on real hardware. Their work decides whether a model…EmergingAI Performance Portability EngineerAI compilers, kernels and runtime systemsAn AI Performance Portability Engineer keeps models fast and accurate across different hardware. They cut dependence on one chip or vendor, which…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← AI accelerators and hardware co-designAll new job families

Start a search

Hiring an AI Accelerator Architect?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.