Hiring nowAI data centre networks and facilities

AI Cluster Network Engineer

Also called ML Networking Engineer

An AI Cluster Network Engineer designs and runs the network that links accelerators during distributed training. A well-built fabric keeps expensive compute busy and lets a job recover quickly when hardware fails.

What they do

They design the network topology for accelerator clusters and decide how racks, switches and links fit together. They read network telemetry to find congestion, uneven traffic and failing links. They run tests that push the fabric the way training jobs do. They work with scheduler owners on job placement so communication stays local where it can. They agree power and cooling limits with facilities teams. When a link or switch fails mid-run, they trace the cause, restore service and change the design or runbook so it is less likely to repeat.

What a strong hire delivers

In the first months, a strong hire gives the team a clear picture of how the current fabric behaves under real training load. They deliver a documented topology and a repeatable set of congestion and availability tests. They fix the worst bottlenecks, shorten fault recovery, and leave runbooks that on-call engineers can follow. Capacity plans for the next expansion are grounded in measured data.

How you can tell they are good

Ask them to walk through a congestion problem they diagnosed, from first symptom to fix. Good candidates name the telemetry they used, the hypotheses they ruled out and how they confirmed the result. Ask how they would test a new topology before real jobs run on it. Listen for clear thinking about collective communication patterns and failure modes, not just vendor feature lists.

Why it is hard to fill

Few people have run networks built for accelerator traffic, so the title is rare and often hidden under other names. Most candidates come from enterprise, carrier or cloud networking and have not met training communication patterns. Others come from HPC and know the workloads but not production operations. Strong people are usually settled, and they weigh scale, ownership and hardware access heavily when they consider a move.

Where to find them

Look at network engineers from large cloud and hyperscale operators, HPC centre engineers, and people who built storage or low-latency fabrics. Titles include network reliability engineer, HPC systems engineer, data centre network architect and ML infrastructure engineer. Some come from electrical or facilities engineering and have moved toward network design. Research labs, model providers and GPU cloud companies are the main places this work happens today.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an AI Cluster Network Engineer

  1. Describe a congestion problem you found in a large network. How did you locate it and prove the fix worked?

    A strong answer names the telemetry used, how they narrowed the cause and how they measured improvement afterwards. Worry if they describe only configuration changes with no evidence, or credit the fix to a vendor without understanding it.

  2. How would you choose a topology for a new accelerator cluster, and what trade-offs would you weigh?

    Look for discussion of bandwidth, oversubscription, cabling, failure domains, growth and cost against job performance. Worry if they give one favourite design for every case or ignore the power and cooling limits of the site.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are building and running large accelerator clusters for training and serving models. We need a network engineer to design the fabric that connects them, keep it healthy under heavy load and make sure failures cost us as little time as possible.

What you will do

  • Design and document network topologies for accelerator clusters
  • Analyse telemetry to find and fix congestion and degraded links
  • Build and run congestion and availability tests
  • Work with scheduler and platform teams on job placement and communication efficiency
  • Agree power, cooling and cabling constraints with facilities teams
  • Lead incident response for network faults and write runbooks
  • Plan capacity for cluster expansion using measured data

What you will bring

  • Demonstrated work designing or operating high-performance network fabrics
  • Experience diagnosing congestion using network telemetry
  • Understanding of distributed training communication patterns
  • Track record of handling production incidents and learning from them
  • Ability to automate tests and configuration with code
  • Clear written and spoken communication with hardware and software teams

Nice to have

  • Experience with HPC interconnects
  • Familiarity with job schedulers and orchestration
  • Exposure to data centre power and cooling design

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Hardware and computer architectureAdvancedDesign computing structures and hardware interfaces
  • Cloud and orchestrationAdvancedProvision and operate scalable compute services
  • Distributed systemsWorkingDesign reliable work across networked machines
  • Reliability and incident responseWorkingDetect failures, restore service and prevent recurrence

Other roles in aI data centre networks and facilities

Hiring nowAI Data Centre Power EngineerAn AI Data Centre Power Engineer plans the electrical capacity and resilience of AI compute sites. They make sure GPU clusters get safe, reliable…Hiring nowAI Thermal Systems EngineerAn AI Thermal Systems Engineer designs and validates the cooling for dense accelerator racks. Good cooling keeps expensive compute inside…Hiring nowAI Compute Scheduler EngineerAn AI Compute Scheduler Engineer decides which model workloads get which cluster resources, and when. Good scheduling keeps expensive GPUs busy…Hiring nowAI Data Centre Programme ManagerAn AI Data Centre Programme Manager delivers gigawatt-scale compute sites, covering land, power, cooling and schedule. They turn a company's compute…Hiring nowAI Power and Energy StrategistThe AI Power and Energy Strategist secures the electricity that AI compute sites run on. Without firm power on a workable schedule, GPUs sit idle…

Related roles in other families

Hiring nowMLOps EngineerMLOps and AI platform engineeringAn MLOps Engineer builds the pipeline that packages, tests and releases models in a repeatable way. It matters because releases become predictable…Hiring nowAI Platform EngineerMLOps and AI platform engineeringAn AI Platform Engineer builds the shared services that let teams use models and build AI applications without each reinventing the plumbing. It…EmergingLLMOps EngineerMLOps and AI platform engineeringAn LLMOps Engineer runs prompt, model and retrieval changes as controlled releases. It matters because language model features break quietly, and…Hiring nowModel Registry and Release EngineerMLOps and AI platform engineeringA Model Registry and Release Engineer keeps every model artefact traceable and every deployment approved. It matters because a company that cannot…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← AI data centre networks and facilitiesAll new job families

Start a search

Hiring an AI Cluster Network Engineer?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.