EmergingPrivacy-preserving and confidential AI

AI Privacy Attack Researcher

Also called AI Privacy Red Teamer

An AI Privacy Attack Researcher tests, with authorisation, whether a model or its data can leak sensitive information. They show the company where exposure is real, so it can be fixed before customers or regulators find it.

What they do

The work starts with a threat model: who might attack, what they can see, and which data matters most. Then the researcher runs membership inference and extraction tests against models and their serving setups, using existing attack suites and custom code where those fall short. They compare results across data slices, because risk is rarely even. They try mitigations, such as changes to training, filtering or access, and retest. Every experiment is logged so someone else can repeat it. Findings go to engineering, security and legal in plain terms, with clear evidence.

What a strong hire delivers

In the first months, a strong hire gives the company a working threat model for its models and data. They produce a reproducible attack harness that the team can rerun after each major change. They deliver a first report showing where leakage is highest by data slice, which mitigations helped and by how much, and what risk remains. They also agree with engineering how privacy testing fits into release decisions.

How you can tell they are good

Ask them to walk through a past privacy attack from threat model to result. Good candidates state their assumptions, explain why a test was or was not meaningful, and admit what they could not prove. Ask how they would show a mitigation actually worked. Weak candidates quote headline attack numbers without baselines, ignore data slices, or cannot explain how someone else would reproduce their work.

Why it is hard to fill

Hardly anyone holds this title. The work often sits inside privacy engineering, security research or ML teams, so CVs rarely say it. The skill mix is uncommon: attack thinking, statistics, and enough ML to run experiments on real models. Many people have two of the three. Strong candidates are often happy where they are, and the best ones are cautious about work that is vague on authorisation and scope.

Where to find them

Look at privacy engineers who have moved towards ML systems, security researchers who have tested models, and cryptographers curious about applied work. Some come from ML research groups that study memorisation and leakage. Others sit in AI red teams, trust and safety teams, or confidential computing groups. Frontier labs, security consultancies, academic labs and larger companies handling sensitive user data are all worth searching.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an AI Privacy Attack Researcher

  1. Describe a membership or extraction test you ran. What was the threat model, and what did you conclude?

    A strong answer names attacker access, data, baselines and the limits of the result. It separates real leakage from noise. Worry if they only cite tools, give a single headline number, or cannot say what the test did not show.

  2. How would you decide which data slices to test first in a model trained on mixed customer and public data?

    Good answers prioritise by sensitivity, duplication, rarity and exposure to attackers. They mention talking to data owners. Worry if they treat all data as equal or propose testing everything with no plan.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We build AI systems that learn from and handle sensitive information. We are hiring a researcher to test, with authorisation, how our models and data could leak, and to show us how to reduce that risk. You will work with engineering, security and legal.

What you will do

  • Build threat models for our models, training data and serving systems
  • Run membership inference and extraction tests and record the results
  • Compare attack success across data slices
  • Test mitigations and measure how well they work
  • Maintain a reproducible attack harness the team can rerun
  • Write clear reports with evidence, limits and recommendations
  • Advise on privacy risk before major releases

What you will bring

  • Demonstrated privacy attack or security testing work on ML models
  • Experience designing experiments with sound baselines and statistics
  • Working knowledge of privacy engineering and security engineering
  • Ability to write reproducible research code
  • Clear written communication for technical and non-technical readers
  • Sound judgement on authorisation, scope and data handling

Nice to have

  • Experience with confidential compute or cryptographic libraries
  • Background in formal verification
  • Work on data quality and provenance

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Privacy engineeringAdvancedLimit disclosure and unauthorised personal-data use
  • Security engineeringAdvancedProtect systems, identities and sensitive assets
  • Formal verificationWorkingProve stated properties under explicit assumptions
  • Data quality and provenanceWorkingTrack dataset origin, validity and transformations

Other roles in privacy-preserving and confidential AI

Hiring nowPrivacy-Preserving ML ResearcherA Privacy-Preserving ML Researcher builds learning methods that limit what a model can reveal about the data behind it. That lets a company train…EmergingConfidential AI EngineerA Confidential AI Engineer runs model training and inference inside attested, restricted execution environments. They let a company use sensitive…Hiring nowAI Privacy EngineerAn AI Privacy Engineer builds consent, minimisation and access controls into AI workflows, so sensitive data is not exposed in training or…

Related roles in other families

EmergingAI Control ResearcherAI control, monitoring and containmentAn AI Control Researcher designs the safeguards that limit what a model can do if it turns out to be unreliable or hostile. They keep risk bounded…EmergingAgent Containment EngineerAI control, monitoring and containmentAn Agent Containment Engineer builds the boundaries that keep AI agents from doing harm, even if the model misbehaves or is attacked. They matter…EmergingAI Monitor Evaluation EngineerAI control, monitoring and containmentAn AI Monitor Evaluation Engineer tests whether the systems that watch model behaviour actually catch dangerous or unauthorised actions. It matters…EmergingAI Shutdown and Recovery EngineerAI control, monitoring and containmentAn AI Shutdown and Recovery Engineer builds and tests the ways to stop an AI system mid-action and bring it back to a known good state. It matters…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← Privacy-preserving and confidential AIAll new job families

Start a search

Hiring an AI Privacy Attack Researcher?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.