A coding agent's final code tells you almost nothing about how it got there. Two agents can complete the same task with identical, review-passing output, while one of them quietly creates artifacts nobody can see and takes destructive actions along the way. The difference lives in the trajectory, not the final artifacts, like code diffs.
This talk argues that defenders should be profiling agent behavior rather than trying to build a perfect box around it, and walks through what happened when we tried to do this. We'll cover why every model needs its own behavioral baseline (there is no generic "coding agent" due to significant differences in models), why "bad" is irreducibly contextual, and the fake company we built- eight employees, a SaaS product, a year of history- to generate replayable trajectories.
The core question: how much lead time do you get before a coding agent session goes wrong? We'll share preliminary results pitting unsupervised anomaly detection against LLM-as-a-judge and rule-based engines, and be honest about what we still can't measure.
About the Presenter: D Niu
D Niu was most recently a Senior Software Engineer at Datadog, focusing on supply chain security and AI threats. D has previously worked on security research, scaling detection systems, and threat hunting capabilities for Datadog Security products as part of security research. D now spends all her time researching coding agent behavior. When she is not building stuff and thinking hard, she is usually running long distances and traveling.
