SEC536: Adversarial AI - Penetration Testing AI Systems


Experience SANS training through course previews.
Learn MoreLet us help.
Contact usBecome a member for instant access to our free resources.
Sign UpWe're here to help.
Contact UsAI failure, the OpenAI/Hugging Face breach, and what it means for the future of cybersecurity

This month, an OpenAI model, running inside a sealed evaluation with its safety limits turned down, found a previously unknown flaw, broke out on its own, and took control of Hugging Face's live systems over a weekend. No human directed it. For years, industry leaders across the field warned this day would come. The disclosures suggest it has arrived.
The harder story is what happened next. When Hugging Face's responders went to investigate, the frontier models they reached for refused to run the analysis, so they pivoted to a self-hosted, open-weight model instead. Offensive research ran unrestricted while the defensive response hit a compliance blocker.
This is a policy story as much as a technical one. SANS faculty and staff, joined by voices from public policy and industry governance, will work through what it changes: AI testing standards, lab resilience, how we model attacker intent, who gets trusted access and who decides, and what defenders should build now, while nothing is on fire.
Watch live on Tuesday, July 28 at 12 p.m. ET. Bookmark this page and join us here when we go live. No registration. No sign-up.

Your next step
AI models have shown they can break out of controlled environments and act entirely on their own, without a human in the loop. During the Hugging Face incident, that's exactly what happened, and the very AI tools built to help investigate ended up standing in the way. Our panel walks through how it unfolded and what it means for anyone whose IR plan assumes their tools will cooperate.
Go deeper with the post-mortem brief SANS co-authored, published by Cloud Security Alliance with contributors from across the industry. Then take the two-minute self-assessment to see if your team is ready to handle an AI-run attack.


Ed Skoudis is President of the SANS Technology Institute, a SANS Fellow, and founder of Counter Hack. As the original author of SEC504 and SEC560, he has shaped modern penetration testing and incident response.
Learn more

Rob T. Lee is Chief AI Officer and Chief of Research at SANS Institute, where he leads research, mentors faculty, and helps cybersecurity teams and executive leaders prepare for AI and emerging threats.
Learn more

Joshua Wright, Senior Technical Director at Counter Hack Challenges and author of SEC504, has spent over two decades teaching and building tools that help defenders identify and counter real-world cyber threats through practical, hands-on learning.
Learn more

James has spent the past 20 years of his life chasing cybercriminals around the Internet and, as a self-professed “massive geek”, has been involved in most cyber security disciplines.
Learn more

Respected as a world leader among public authorities for cybersecurity, Ciaran Martin is the UK’s National Cyber Security Centre’s (NCSC) founder and former head.
Learn more

Rich is the Chief Analyst at the Cloud Security Alliance where he focuses on leading-edge cloud and AI security research and implementation.
Learn moreOn Tuesday, SANS hosted a live community panel on the OpenAI / Hugging Face breach, titled "The Sandbox Let It Out. The Guardrails Locked Us Out.", in partnership with the Cloud Security Alliance.
Expedited Strategy Briefing By the CSA CISO Community, SANS, [un]prompted, RSAC, Knostic, FIRST, and the wider community. The Cloud Security Alliance's CISO community, working with the Hugging Face response team, has published an initial post-mortem on the July breach in which two OpenAI models escaped a sandboxed evaluation and compromised Hugging Face production infrastructure.
Rob T. Lee on what the Hugging Face post-mortem changes for incident response.
An AI broke out of its own test and hit a live company. The defenders who came to clean up were the ones the guardrails stopped. Here is why we are pulling the community together to talk about it.
On July 16, Hugging Face published an article on an intrusion they suffered on their production infrastructure.