AI Vulnerability Discovery Is Already Here. We Have Been Doing It for 15 Months.

Since Anthropic announced Mythos, I have had three friends with Mythos access reach out to me to discuss how they are using it and to brainstorm additional ways to leverage it. I have had conversations with bank CISOs trying to figure out what to tell their boards about Mythos and this next generation of AI models. And I have spent the last week watching the online conversation swing between “This is the end of cybersecurity!” and “This is only pure-play marketing hype!”

Both of those takes are wrong. I think I can explain why, because my team has been living in the middle of this for a while now.

My team of sixteen penetration testers has been doing AI-enabled, source-assisted penetration testing for 15 months, but not with Mythos. Instead, we’ve been using current models you can access right now. When the Mythos numbers came out, I wasn’t shocked. Instead, it helped confirm what my team has been doing and seeing.

What We Have Seen

Source-assisted penetration testing is something my team has done for 15 years. That’s where we perform a pen test with source code in hand. We wouldn’t review the source code for flaws directly (that’s another kind of work – source code review). Instead, we’d perform the penetration test itself, and whenever we noticed behavior that looked like a potential vulnerability, we’d then check the source code to gain insight into whether we had found a bug that could be exploitable. We’d then create the exploit (by hand) and verify that it worked so we could report where in the code the flaw is and make suggestions on how to fix it. It’s good work and has proven to be helpful. 

But fifteen months ago, my team flipped the workflow with AI-enabled source-assisted penetration testing. For some projects, we start with the source code. We then have an AI agent work through the code to find potential flaws. We carefully prompt the AI to weed out hallucinations and other false positives. And then, we have the AI create exploits and launch them at the target, all under human supervision, of course.

The results have been hard to process, honestly. We will do a pen test on a system we tested the year before, where a really skilled human already went through it. And then on day one, using this methodology, we find five critical findings. That’s commonplace now, and the number of findings in our penetration tests has mushroomed upwards as we find more and more critical and high-risk flaws using AI-enabled source-assisted penetration testing.

These are not exotic new vulnerability classes. We have found authentication bypasses, authorization flaws, BOLA, IDOR, cross-tenant access, race conditions, and more. All are well-known categories. But the AI finds them in places humans may not look, such as subtle flaws in obscure workflows that put the system into an unusual state where something is exploitable.

The Workflow

During our BugBusters broadcast this week, SANS Instructor and penetration tester Chris Elgee walked the audience through the end-to-end process on screen against DataEase, an open-source business intelligence (BI) solution with about 1.7 million lines of code, with much of its UI in Chinese. Good luck doing that manually. But the AI did not care.

Chris primarily used GPT-5.4 with some Opus 4.6 for validation, but we swap models constantly. It’s super important to point out that any workflow locked to a single model has a single point of failure. Instead, I always advise people to ensure their AI-enabled workflows (for whatever purpose) are built to allow for seamless interchange of underlying models. That’s because sometimes models degrade over time, refuse certain requests as guardrails change, or sometimes (truly) a model is just having a bad day. Seriously.

Here is how our AI-enabled, source-assisted penetration testing process works:

Map the codebase. Have the AI generate a comprehensive summary of every function, dependency, and workflow. This becomes its reference document for the entire engagement. No model can process 1.7 million lines at once, so context window management is the entire game, as we chunk code into different pieces.

Identify the high-value 20%. Authorization flows, input validation, session management, and OAuth handling are all potentially rich areas of attack. Have the AI generate a candidate matrix of potential flaws. The discipline here is keeping the model from jumping ahead to “here is a critical vulnerability” before it has reliable results. Constrained prompting helps keep the agent focused on the task at hand.

Narrow progressively. Each prompt focuses the limited context window on finding real flaws. We move from the full(-ish) repo to a candidate vuln matrix to a specific function to focused dependencies and data flows. This is the step most people skip, and it is the reason most people get garbage results and say AI does not work for vulnerability discovery.

Build a test harness. When you have a promising candidate, have the AI extract the relevant potentially vulnerable code and build a local test environment. Chris asked the model to do this, and it pulled the necessary components from 1.7 million lines and built a working harness in Python that mocked up the vulnerable code in a running state in about a minute.

Exploit and validate. Instruct the AI to build an exploit and tell it not to overstate its results. The model has to prove the exploit works. Chris’s demo ended with unauthenticated write access to the web root to get remote code execution on the system, which the model claimed was ranked as CVSS 9.6. We then used the exploit against an actual target system (not the mock-up test harness), and it was successful.

Reporting. We then prompt the AI to create a report of its findings, giving it an example format for normal penetration testing findings. The reports are typically quite good but require a little human fine-tuning to ensure they properly explain the business risk and appropriate remediation steps.

Bring the human. Every finding needs human validation. Every exploit needs human review. As Chris said during the broadcast, “It is artificial intelligence. You do have to bring your actual intelligence to the game here.”

One note on ethics: before you upload any source code to a model, confirm you have permission. Check your NDA, your client agreements, and your internal data classification policies. If any of that is unclear, use a local model or a zero-retention enterprise tenant. Every time.

What Comes Next

Josh Wright closed the broadcast with some reflections on the implications of these capabilities for the cybersecurity industry more generally. His framing has been on my mind since.

A researcher at Anthropic recently said that the last twenty years in cybersecurity have been “relatively stable.” Josh's first instinct was to push back. But the more he thought about it, the more he realized this was right. We have had a manageable cadence of new vulnerabilities, a small number of people capable of writing zero-day exploits, and patching processes designed around that tempo. That is changing quickly.

Josh’s estimate is that we will face an onslaught of twenty to forty times the current volume of new vulnerabilities within the next year. And there is a second wave most people have not considered. Right now, AI vulnerability discovery focuses on open-source software. But open-source components are embedded in everything, including commercial closed-source solutions, which will inherit the flaws of their open-source components. Compounding the problem, within 6 to 12 months, Josh expects AI models to get meaningfully better at reverse engineering closed-source software binaries directly. That will mean another spike in AI-discovered vulns without source code available at all.

The short-term picture is uncomfortable. Nation-states sitting on expensive zero-days now have an expiration date on them courtesy of the new models. Expect accelerated exploitation as stockpiles get burned.

But look at the named partners of Anthropic’s Glasswing initiative to leverage Mythos to find flaws before a general Mythos release: AWS, Apple, Microsoft, Linux kernel maintainers, Cisco, and more. Imagine those platforms with their major vulnerability backlogs cleared and patched. One to three years out, the baseline of software security could be better than anything we have seen. Josh put it well: “There is a glimmer somewhere for a much more secure planet.” The path there is going to be rough. But the destination is real.

What I Think You Should Do

Vulnerability management, as most organizations practice it, was built for a handful of new zero-days per week or month. That world is ending. Organizations need to explore building a permanent VulnOps capability, staffed and automated for continuous discovery of flaws and patching the most important ones.

Patching needs to get better at every step, not just faster deployment. Creation, distribution, testing, and validation can all improve for most organizations.

Do not lose the fundamentals either. Keep a careful watch on privilege management, blast radius reduction, and mean time to detect. These controls limit what an attacker can do after they get in, and in a world where exploitation is faster, that time matters immensely.

And start using these tools yourself. You do not need Mythos to find flaws in your own in-house developed code. Current models work. We released the SEC543 Field Guide this week as a free one-page cheat sheet covering this workflow with tips and hints we’ve learned over the past 15 months. If you want the full hands-on course, SEC543 launches June 1-2.

Josh told the audience something worth repeating: never let a good crisis go to waste. This is an opportunity for every security practitioner to become the person in their organization who understands what is happening and can translate it into action. The community that shows up and builds the defensive capability is the one that gets us to the other side.

Resources

Full BugBusters replay — The complete one-hour session where Chris demonstrates AI-assisted vulnerability discovery end-to-end against 1.7 million lines of code, from mapping the repo through validated exploitation. His demo starts around the 8-minute mark.

SEC543 Field Guide: Tips for Using AI for Source Code Vulnerability Analysis — A free one-page reference covering the five-step discovery loop, prompt patterns that actually produce findings, hallucination defenses, and the legal checkpoints you need to clear before uploading source code to any model.

SEC543: AI-Assisted Source Code Analysis and Exploitation for Penetration Testers — The full two-day hands-on course built from the methodology you saw in the broadcast. No software development experience required. The AI reads the code while your team provides the security expertise. June 1-2.

The AI Vulnerability Storm: Building a Mythos-Ready Security Program — The CSA/SANS joint briefing written by Gadi Evron (CEO, Knostic and CISO-in-Residence for AI at CSA), Rich Mogull (Chief Analyst, CSA), and Rob Lee (Chief AI Officer and Chief of Research, SANS Institute) with 60+ contributors including former CISA Director Jen Easterly, Bruce Schneier (Fellow, University of Toronto), and former National Cyber Director Chris Inglis. Includes a 13-item risk register, 11 priority actions with start dates, and a board-ready executive briefing section. If you read one document this month, make it this one.

SANS AI Cybersecurity Summit, April 20-21 — Two days of hands-on sessions on defending against AI-powered threats and deploying AI securely. Keynotes from Bruce Schneier; Jacob Klein, Head of Threat Intelligence at Anthropic; Anne Neuberger, Senior Advisor at a16z; Sounil Yu, Co-Founder and Chief AI Safety Officer at Knostic; Diana Kelley, CISO at Noma Security; and Pliny the Liberator, AI Red Teamer at BT6 Collective, among others. Free for live online access.

Find Evil! SANS AI Hackathon — SANS has launched Find Evil!, the first hackathon for autonomous AI incident response, and 1,400+ people have already joined. The only way to keep pace with AI-driven attacks is AI-driven defense, and the community that built the SIFT Workstation over 19 years is the one that can perfect Protocol SIFT, so every practitioner can deploy it. Two months, $22,000+ in prizes, all skill levels. Join us!