Hiring nowAI application security and misuse defence

AI Trust and Safety Specialist

Also called AI Trust & Safety Specialist · Model Policy Enforcement Analyst

An AI Trust and Safety Specialist writes and enforces the rules for how a model-connected product can be used, and investigates misuse. They keep the company out of harm's way while legitimate users carry on working.

What they do

They draft usage policies in language that engineers, support teams and users can all act on. They review reported and flagged misuse cases, decide what happened, and apply the policy consistently. They work with security and engineering to turn recurring abuse patterns into detections and product controls. They test the system's limits, sometimes alongside red teamers, to see where the policy fails in practice. They write up incidents, track repeat offenders and tell the team when a rule is not working. They also handle appeals and edge cases that the policy never anticipated.

What a strong hire delivers

In the first months, a strong hire gives the company a usage policy that is clear and enforceable, plus a consistent way to triage and decide misuse cases. They set up reporting that shows which abuse patterns are growing. They agree escalation routes with security, legal and support. They also get at least a few repeat abuse patterns turned into automated detections or product changes.

How you can tell they are good

Ask them to walk through a real misuse case from first signal to final decision. Strong people explain the evidence they used, the call they made and what they changed afterwards. Ask them to critique a short policy and find the loopholes. Listen for specifics about tool use, agents and indirect abuse. Be wary of answers that are all principle and no examples.

Why it is hard to fill

The title is used loosely. Some companies mean content moderation, others mean security investigation, others mean policy writing. Few people have done all three against model-connected products. Candidates tend to be strong in one area and thin in the others. Demand is steady from labs and platforms, so good people are rarely looking. Clarify which mix you need before searching.

Where to find them

Many come from application security, threat detection or security research, and have moved towards model misuse. Others come from trust and safety operations, policy or investigations teams at platforms, marketplaces and social products. Some have worked in fraud or abuse teams. Labs and model-based platforms employ the most experienced people, so look at who has handled live misuse, not only who has written policy.

Indicative pay

Not enough published evidence yet to give a fair range for this role, which is common for work this new. We benchmark it for you as part of a search.

Interview questions for an AI Trust and Safety Specialist

  1. Tell us about a misuse case you investigated. What was the first signal, and what did you decide?

    A strong answer shows evidence gathering, a clear decision and a follow-up change to policy or detection. Worry if the story is vague, if they cannot say what they would do differently, or if they only escalated it.

  2. How do you write a usage policy that engineers can build against and users can understand?

    Good answers use concrete examples, defined terms, clear thresholds and a review process with engineering. Worry about long, abstract principles, no mention of enforcement, or policy written without input from the people who apply it.

4 more questions, with what a strong answer covers, are in the Role Kit.

Job description you can post

We are hiring an AI Trust and Safety Specialist to own how people may and may not use our model-powered product. You will write our usage policies, investigate misuse and work with security and engineering to reduce abuse while keeping the product useful for genuine customers.

What you will do

  • Write and maintain clear, enforceable usage policies
  • Review and decide misuse cases consistently
  • Investigate abuse patterns and report on trends
  • Work with engineering to turn findings into detections and product controls
  • Test the product for misuse routes, including through connected tools
  • Manage escalations, appeals and incident write-ups

What you will bring

  • Demonstrated work investigating abuse or misuse in a live product
  • Experience writing policy or decision guidance that others applied
  • Background in application security, threat detection, security research, or trust and safety operations
  • Sound judgement on ambiguous cases, with clear written reasoning
  • Ability to work with engineers on detection logic and controls
  • Clear written and spoken communication with non-specialists

Nice to have

  • Experience with AI red teaming
  • Familiarity with agent and tool integrations
  • Incident response experience

Free to use and adapt. If it saves you time, a link back to this page is appreciated.

Skills that matter

  • Security engineeringWorkingProtect systems, identities and sensitive assets
  • AI red teamingWorkingTest systems adversarially under authorisation
  • Tool and protocol integrationWorkingConnect models to external data and capabilities
  • Reliability and incident responseWorkingDetect failures, restore service and prevent recurrence

Other roles in aI application security and misuse defence

Hiring nowAI Security EngineerAn AI Security Engineer hardens the applications, interfaces and sensitive assets built around models. They find how attackers could misuse the…Hiring nowAI Red Team EngineerAn AI Red Team Engineer runs authorised attacks against model-connected systems to find weaknesses before real adversaries do. They give the company…Hiring nowAI Abuse Detection AnalystAn AI Abuse Detection Analyst spots unauthorised and harmful use of a company's models and products, then turns what they find into rules and cases…EmergingModel Supply Chain Security EngineerA Model Supply Chain Security Engineer protects model files, datasets and dependencies from tampering. They give the company confidence that what it…Hiring nowAgent Security EngineerAn Agent Security Engineer protects AI agents from prompt injection, data leaks and tool misuse. They let a company ship agents that act on real…Hiring nowModel Weights Security EngineerA Model Weights Security Engineer protects trained model weights and the training infrastructure around them from theft, including by well-resourced…Hiring nowAI Threat Intelligence AnalystAn AI Threat Intelligence Analyst tracks how threat actors use AI to attack, deceive and abuse systems, then turns what they find into detections…EmergingAgent Identity and Permissions ArchitectThis person designs how AI agents prove who they are and what they may do. They keep access narrow as agent numbers grow, so one compromised agent…

Related roles in other families

Hiring nowCybersecurity ML ScientistCyber defence and security operations AIA Cybersecurity ML Scientist builds and tests models that spot malicious activity in authorised security data. They keep detection useful as…Hiring nowAI Detection EngineerCyber defence and security operations AIAn AI Detection Engineer builds the detections that turn AI-derived signals into alerts a security team can trust. They cut noise, catch real…EmergingAI Security Operations EngineerCyber defence and security operations AIAn AI Security Operations Engineer builds constrained AI support for security triage and response. It lets a security team move faster on alerts and…EmergingDefensive AI Agent EvaluatorCyber defence and security operations AIA Defensive AI Agent Evaluator tests how AI agents behave inside security detection and response work, and shows where they fail. It matters because…

How we wrote this

Role definitions are our own editorial work, checked against occupational classifications, standards, research and employer pages. Titles vary by company, and this is a description of the work, not a job advert.

← AI application security and misuse defenceAll new job families

Start a search

Hiring an AI Trust and Safety Specialist?

Tell us the work. We find the people already doing it, whatever their job title says, and you meet them already interviewed for your role.