Every team shipping an LLM guardrail hits the same wall twice. First at the start, when you need to train a classifier to catch harmful prompts but have almost no labeled attacks to learn from. Then again in production, when the model is live but real attacks arrive rarely and unpredictably, so you have no steady supply of fresh failures to keep it sharp. A guardrail that stops improving the day it ships is a guardrail attackers eventually walk through.
This talk walks through how we built and continuously harden a multi-task prompt-security model, one that predicts whether a prompt is harmful, what its underlying goal is, and what framing it uses to disguise that goal, and treats the guardrail as something to attack like any other control.
We start with the cold-start problem: with no attack corpus, we generate synthetic harmful prompts across a structured goal-and-framing taxonomy, with an automated judge to keep the labels honest, and bootstrap a working classifier from nothing.
Then we close the production gap. Instead of waiting for real attacks, we run adversarial agents that continuously generate novel attacks, wrap them in stacked jailbreak framings, and push them through the live model to surface what slips past. Verified failures, and the legitimate prompts the model wrongly blocks, feed a gated retrain loop: a new model ships only if recall holds and false positives stay within budget. We show what the red-teaming actually finds, including confident misses and over-refusals the offline metrics hide, and the false-positive triage problem that comes with machine-generated findings at volume.
Attendees leave with a concrete, reproducible recipe for standing up a safety classifier with no data and keeping it honest with adversarial automation rather than hope.
About the Presenter: Rajesh Khanna Bolloju
Rajesh Bolloju is a data scientist on the AI security team at Sophos, where he builds machine learning defenses for large language model and agent security. His work focuses on detecting prompt injection and jailbreaks, automated adversarial red-teaming of safety classifiers, and representation-level analysis of how attacks show up inside a model. This talk is joint work with Adarsh Kyadige, who designed the multi-task prompt-security model at its center.
