Your Agent’s Vulnerability Patches are Probably F.L.A.W.E.D.
A significant and increasing portion of code contributions today come from LLMs. With the announcement of initiatives such as Anthropic's Project Glasswing and OpenAI's Project Daybreak, security patches are no exception to this trend. Today, both AI-assisted and fully AI-generated patches are making their way into codebases everywhere, with as-yet-unknown long term consequences.
Our talk presents research on the efficacy of frontier LLMs at patching novel vulnerabilities based on realistic bug reports. How many LLM-generated patches are actually FLAWED—Fix-Like Artifacts with Embedded Defects? It turns out, about three-quarters of them! We gauge success based on a model’s rate of fully mitigating the original vulnerability, as well as whether an LLM erroneously modifies application behavior and even introducing new vulnerabilities. We will showcase FLAWED, our recently-published testing toolchain that allows for evaluating arbitrary models’ patching capabilities at scale. FLAWED ingests vulnerability descriptions and runs models through a gauntlet of varying prompts, execution modes, and environmental constraints over numerous iterations, tasking different LLMs to fix each vulnerability and evaluating their performance. We will present our findings from a dataset of over 6,000 patching runs produced through FLAWED. We will also demonstrate a variety of real-world case studies in which AI-generated patches introduced vulnerabilities to major open-source projects, including the Linux kernel.
Our data strongly suggests that the adoption of highly-automated, LLM-based patching pipelines should be treated with serious caution, and FLAWED gives organizations a way to quantify the risk inherent in doing so.
About the Presenter: Spencer Michaels
Currently working on cutting-edge security research at 1Password's Off-by-1 Labs (0b1), Spencer brings many years of experience as a former Principal Security Engineer at Trail of Bits, where he performed complex security audits for Fortune 100s, startups, and many FOSS projects. He previously served as a Research Director at NCC Group, presenting landmark research on unikernel security and VMI debugging. His areas of interest include virtualization, Linux, and all things low-level. Spencer believes that in an age of unprecedented puffery and speculation in our field, we need rigorous, empirically-grounded security research more than ever.
