Most LLM security failures are discovered by attackers, not engineers, and the gap between those two events is getting shorter.

Manual prompt testing is how serious vulnerabilities get missed

Security teams currently test LLM behavior by hand, cycling through ad hoc prompts with no structured coverage and no repeatable scoring. The result is an incomplete picture of model exposure that no one can defend against on a deadline.

garak turns that guesswork into a documented, reproducible audit

NVIDIA garak Tutorial lets you run structured red-team scans against any LLM by chaining probes, detectors, and generators into a single configurable workflow. You install the package, point it at a model, and receive a scored report with AVID-compatible vulnerability exports. The real surface area comes from writing custom probes and detectors that match your specific deployment risks, not just the defaults.

Security engineers are the first to need this

• ML security engineers who need evidence of pre-deployment testing for compliance sign-off
• Red team leads who waste hours writing one-off jailbreak scripts with no audit trail or scoring
• AI platform architects who need a repeatable benchmark before promoting a model to production

The plugin discovery and dry-run features mean teams can validate probe coverage before burning compute on a full scan.

LLM auditing is becoming a procurement requirement, not a best practice

The EU AI Act’s high-risk system requirements are forcing enterprise buyers to ask vendors for documented adversarial testing results, a bar that informal prompt logging cannot meet. Teams without a structured red-teaming framework are already behind organizations using garak or comparable tooling to generate defensible audit artifacts.

What you can build with it right now

• Run multi-probe scans across an entire model and export scored vulnerability reports
• Write custom probes targeting domain-specific risks your default suite ignores
• Build custom detectors to flag model outputs against your own policy definitions
• Export findings in AVID format for integration with existing security workflows

Each scan produces a report filename you can pipe directly into downstream triage tooling.

Pricing

garak is open source under NVIDIA’s Apache 2.0 license, free to use and extend.

The honest tradeoff

garak requires Python comfort and meaningful setup time to write custom probes that reflect real production risks, so teams without ML engineering support will plateau at the default scan coverage.

If you need alternatives

Microsoft’s PyRIT covers similar adversarial testing ground with tighter Azure integration. Promptfoo takes a lighter-weight, config-file approach aimed at developers who want fast CI pipeline checks over deep security audits.

LLM red-teaming is moving from optional to contractually required

We cover tools like this every Friday — subscribe here and we’ll send the best ones straight to you.