Group Purchasing
Group Purchasing
AI-FOCUSEDNEW

SEC536: Adversarial AI - Penetration Testing AI Systems

SEC536Offensive Operations, Artificial Intelligence
  • 3 Days (Instructor-Led)
  • 18 Hours
Course authored by:
Foster NethercottMick Douglas
Foster Nethercott & Mick Douglas
SEC536: Adversarial AI - Penetration Testing AI Systems
Course authored by:
Foster NethercottMick Douglas
Foster Nethercott & Mick Douglas
  • GIAC AI Penetration Tester (GAIPT)
  • 18 CPEs

    Apply your credits to renew your certifications

  • In-Person or Virtual

    Attend a live, instructor-led class from a location near you or virtually from anywhere

  • Intermediate Skill Level

    Course material is geared for cyber security professionals with hands-on experience

  • 10 Hands-On Lab(s)

    Apply what you learn with hands-on exercises and labs

Learn the attacks real-world adversaries are executing against enterprise AI deployments spanning LLMs, RAG pipelines, ML models, and vision systems.

Course Overview

Every organization wants AI features. Very few are asking what those features look like from an attacker's view. Teams are skipping foundational security work and building systems that fail in ways traditional appsec tools were never designed to catch. SEC536 puts you in the attacker's seat so you can see exactly how those failures happen.

Your Organization Is Deploying AI Systems, Whether Security Is Ready or Not

Most organizations are deploying AI faster than they are securing it. The attacks in this course map directly to vulnerabilities being found in production systems and exploited by attackers today. By emulating them accurately against a realistic target, you give your organization the only thing that informs a security investment: an honest measure of its real risk exposure.

You will learn to attack AI systems the way real adversaries do today through hands-on exploitation of RAG-backed assistants, prompt injection techniques, agent abuse, model weight theft and manipulation, model poisoning, and MCP server attacks. Each technique is paired with clear defensive insights, mapping what you just broke to the architectural decisions, controls, and detection opportunities that would have stopped it, so you leave not just knowing how attacks work, but where and how to stop them in real environments.

What You’ll Learn

  • Chain indirect prompt injections across agents to steal production weights
  • Exploit RAG retrieval boundaries and outbound channels to exfiltrate sensitive documents
  • Defeat facial recognition and identity checks with adversarial attacks and patches
  • Attack LLM APIs through role confusion, path traversal, and missing authentication
  • Compromise agentic systems via dynamic tool discovery, parameter abuse, and context injection
  • Compromise MCP servers through techniques such as SQL injection, homoglyph tool shadowing, and name collision

Business Takeaways

  • Assess AI Systems against the attacks they actually face
  • Counter AI-specific threats beyond the OWASP top 10
  • Reduce AI deployment risk before the first incident
  • Validate vendor claims with attacker side evidence
  • Protect model weights, training data, and AI supply chains
  • Map attack techniques to architectural and detection controls

Course Syllabus

Explore the course syllabus below to view the full range of topics covered in SEC536: Adversarial AI - Penetration Testing AI Systems.

Section 1Foundations of Attack Techniques, AI Reconnaissance, and Prompt Injection

In this section, we establish how AI models process input, then use that understanding to locate live deployments and subvert their instructions. Coverage moves from model internals through AI-specific reconnaissance to direct prompt injection.

Topics covered

  • AI components, emergent properties, and misalignment as attack surface
  • How LLMs process input: tokenization, embeddings, attention, and sampling
  • Frameworks and case studies: OWASP LLM Top 10, MITRE ATLAS, EchoLeak
  • AI-specific reconnaissance, stack mapping, and model fingerprinting
  • Direct prompt injection, prefill, role confusion, and delimiter escapes

Labs

  • Lab 1.1: Abusing AI Assistants
  • Lab 1.2: Reconnaissance Against AI Systems
  • Lab 1.3: Prompt Injection
  • Mini Lab: Emergent issues, Alignment Problems, Bias, Architectural Flaws

Overview

Attacking an AI system starts with understanding why it fails.

Section 1 establishes how models process input (tokenization, embeddings, attention, and sampling) and shows how each of those mechanisms becomes an attack primitive rather than an implementation of detail. You will learn why the instruction/data boundary that protects traditional applications was never designed to exist in a language model, and why safety alignment is a probability adjustment rather than a control.

From there you move into reconnaissance. AI deployments announce themselves through exposed inference endpoints, orchestration frameworks, vector databases, model registries, and observability dashboards. You will fingerprint the full AI application stack from the outside in, identify the model behind an API, discover vision systems, and map every surface where untrusted content can reach a context window.

The section closes with direct prompt injection: attention capture, instruction override, prefill and context injection against API message arrays, ChatML and application delimiter escapes, role confusion, and function schema poisoning.

Section 2Jailbreaking, Alignment Exploitation, and Defense Bypass

Guardrails are behavioral, not architectural. This section covers the jailbreak technique tree, the alignment failures that make it work, and how to defeat the input and output filters wrapped around production deployments.

Topics covered

  • How safety training works and why jailbreaking succeeds
  • Jailbreak techniques: personas, logic traps, framing, and policy puppetry
  • Context manipulation: flooding, positional exploitation, attention hijacking
  • Alignment exploitation: reward hacking, sycophancy, and sandbagging
  • Constitutional AI attacks, defense fingerprinting, and filter bypass

Labs

  • Lab 2.1: AI Architecture Flaws
  • Lab 2.2: Jailbreaks and Weight Theft
  • Lab 2.3: AI Alignment Issues
  • Lab 2.4: Computer Vision — Evasion and Adversarial Attacks
  • Mini Lab: Alignment Problems

Overview

Guardrails are behavioral, not architectural.

Section 2 is about moving the output distribution far enough that refusal stops being the most likely next token. You will work through the jailbreak technique tree: persona depth, logic traps, hypothetical framing, Bad Likert Judge, persuasion patterns, chain-of-thought corruption, policy puppetry, context flooding, positional exploitation, and attention hijacking. For each, you will learn the measured success rate behind the technique.

You will then attack alignment itself. Specification gaming, reward hacking, goal misgeneralization, sycophancy, alignment faking, and sandbagging are not abstract safety research problems; they are testable failure modes in production deployments, and this section gives you the probe methodology for each. Constitutional AI receives dedicated treatment through principle inference, principle conflict, scope abuse, and constitution parroting.

The section closes on the defense stack: identifying which layer blocked you, fingerprinting commercial guardrail products, and defeating input and output classifiers through multi-language attacks, token smuggling, semantic reframing, encoding, obfuscation stacking, MathPrompt, ASCII art, and payload splitting.

Section 3Indirect Injection, Infrastructure, and Agentic Systems

This section covers payload delivery through documents and retrieved content, the infrastructure and API layers surrounding the model, and agentic systems where a single injection inherits every capability the agent holds.

Topics covered

  • Indirect injection: lethal trifecta, documents, email, and web content
  • RAG pipeline poisoning and canary token methodology
  • Infrastructure exposure, side channels, and weight extraction
  • API security: credentials, authorization layers, and denial-of-wallet
  • Agentic attacks, multi-agent topologies, and MCP exploitation

Labs

  • Lab 3.1: AI Architecture Flaws
  • Lab 3.2: Exploiting Mass Assignments and Hidden Features
  • Lab 3.3: Attacking Agentic AI
  • Lab 3.4: Attacking MCP Servers

Overview

The most consequential AI attacks never touch the chat box.

Section 3 opens with indirect injection: planting payloads in documents, spreadsheets, presentations, email bodies, calendar invites, images, audio, and retrieved web pages, then letting the target's own AI retrieve and execute them. You will work through the lethal trifecta and the confused-deputy problem, understand why WAF, DLP, and EDR see nothing, and poison a RAG pipeline with a handful of documents against millions of legitimate ones.

Attention then turns to the infrastructure and integration layers. Timing side channels and logit leakage recover architecture details and model weights. Exposed Ollama hosts, MLflow tracking servers, Weights and Biases keys, and observability dashboards hand over training artifacts and full prompt histories. Provider credential shapes, layered authorization models, streaming response injection, and recursive agent fan-out expose the API surface, including denial-of-wallet.

The section closes on agentic systems, where a single injection inherits every capability the agent holds. You will attack multi-agent topologies, inject plans and manipulate goals, exploit inter-agent trust and impersonation, compromise orchestrators, test memory isolation across tenants and sessions, and exploit MCP servers through tool poisoning, homoglyph shadowing, name collision, and rug-pull mutation.

Things You Need To Know

Important! Bring your own system configured according to these instructions!

A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will likely leave the class unsatisfied because you will not be able to participate in hands-on exercises that are essential to this course. Therefore, we strongly urge you to arrive with a system meeting all the requirements specified for the course.

It is critical that you back-up your system before class. It is also strongly advised that you do not bring a system storing any sensitive data.

CPU

  • 64-bit Intel i5/i7 2.0+ GHz processor
  • CRITICAL NOTE: Apple Silicon devices cannot perform the necessary virtualization and therefore cannot be used for this course.
  • Your system's processor must be a 64-bit Intel i5 or i7 2.0 GHz processor or higher. To verify on Windows 10 or 11, press Windows key + "I" to open Settings, then click "System", then "About". Your processor information will be listed near the bottom of the page. To verify on a Mac, click the Apple logo at the top left-hand corner of your display and then click "About this Mac".

BIOS

  • Enabled "Intel-VT"
  • Intel's VT (VT-x) hardware virtualization technology must be enabled in your system's BIOS or UEFI settings. You must be able to access your system's BIOS to enable this setting in order to complete lab exercises. If your BIOS is password-protected, you must have the password. This is absolutely required.

RAM

  • 16 GB RAM is highly recommended for the best experience. To verify on Windows 10, press Windows key + "I" to open Settings, then click "System", then "About". Your RAM information will be toward the bottom of the page. To verify on a Mac, click the Apple logo at the top left-hand corner of your display and then click "About this Mac".

Hard Drive Free Space

  • 100 GB of FREE space on the hard drive is critical to host the VMs and additional files we distribute. SSD drives are also highly recommended, as they allow virtual machines to run much faster than mechanical hard drives.

Operating System

  • Your system must be running either the latest version of Windows 10, macOS 10.15.x or later, or Linux that also can install and run VMware virtualization products described below.

Additional Software Requirements

VMware Player Install:

  • Download and install VMware Workstation Pro 17+ (for Windows hosts), or VMWare Fusion Pro 13+ (for macOS hosts) prior to class beginning. Workstation Pro and Fusion Pro are now available free for personal use from the VMware website. Licensed commercial subscriptions to these products can also be used.
  • Other virtualization products, such as Hyper-V and VirtualBox, are not supported and will not work with the course material.

Your course media will now be delivered via download. The media files for class can be large, some in the 40 - 50 GB range. You need to allow plenty of time for the download to complete. Internet connections and speed vary greatly and are dependent on many different factors. Therefore, it is not possible to give an estimate of the length of time it will take to download your materials. Please start your course media downloads as you get the link. You will need your course media immediately on the first day of class. Waiting until the night before the class starts to begin your download has a high probability of failure.

If you have additional questions about the laptop specifications, please contact customer service.

  • SEC536 is built for security practitioners who interact with and need to assess AI systems the way they work, not the way a demo describes them.
  • Penetration testers and red teamers get techniques that work against LLM-integrated targets instead of recycled web app attacks with an AI label.
  • Application security engineers, architects get the threat models and testing patterns they need as AI features hit production.
  • Data scientists and ML platform engineers get an unfiltered look at how their pipelines and deployed models appear from an attacker's perspective.
  • Cyber defenders and blue teamers get to anticipate attacks instead of reacting to the first incident.
  • Technical managers get ground truth that cuts through vendor hype and shows where AI risk lives in the stack.

The GIAC AI Penetration Tester (GAIPT) certification validates a practitioner's hands-on AI penetration testing skills for assessing and exploiting vulnerabilities in enterprise AI systems.

  • Direct and indirect prompt injection
  • RAG exploitation
  • Agentic systems
  • Alignment problems
  • MCP server attacks
  • API security
  • AI Architectural flaws

More Certification Details

You keep a VM containing ten hands-on labs. Run them repeatedly, try hard mode, or any of the multiple bonus modes each lab has. You also leave with the Marimo workbooks covering every major concept and a working mental model of how AI-integrated systems fail and how to communicate that to the people funding the rollout.

  • Unlimited access to all hands-on lab exercises that never expires
  • Printed and electronic course books and a hands-on workbook
  • MP3 audio files of the entire course
  • Detailed video walkthroughs for all lab exercises
  • Visual association maps to break down complex material
  • A digital index for quick reference to all material
  • Bonus content and hands-on exercises to develop your skills beyond the course
  • Essential cheat sheets for tools and complex analysis tasks

Students should be comfortable with HTTP APIs and familiar with general web and application security concepts. An AI or ML background is helpful but not required. Python experience is useful for customizing payloads and tooling but is not a prerequisite.

Start with SEC504: Hacker Tools, Techniques, and Incident Handling to develop an attacker mindset, then SEC560: Enterprise Penetration Testing to learn the methodology for conducting pen tests and the full attack lifecycle, then this course, SEC536, to specialize in attacking AI-integrated systems, then SEC660: Advanced Penetration Testing, Exploit Writing, and Ethical Hacking to learn how to level up your skills and develop novel, advanced techniques.

SEC536: Adversarial AI is built around a single discipline: studying and executing the attacks that adversaries are running against production AI systems today. Adversarial AI is the offensive side of AI security, where penetration testers, red teamers, and security engineers exploit LLMs, RAG pipelines, agents, MCP tool servers, and the model supply chain to surface failures that traditional security tooling was never designed to find. Its purpose is to give organizations an honest picture of their AI risk exposure so they can make informed decisions about the AI features they are racing to deploy.

  • Surfaces AI-Specific Failure Modes: AI systems break in ways that do not map cleanly to the OWASP Top 10 or established application security playbooks. Adversarial techniques bring these failures into view before attackers find them, spanning prompt injection, indirect injection, jailbreaks, evasion, model poisoning, weight extraction, and agentic and MCP abuse.
  • Measures Risk Before the First Incident: Emulating real attacks against production-shaped systems gives organizations a measured view of where their AI rollouts will fail. That measurement is at the core of any meaningful security investment, and it must come before something breaks publicly, not after.
  • Replaces Demo with Evidence: Most AI security claims fall apart under sustained attacker pressure. An adversarial assessment produces concrete evidence, so security teams can evaluate AI products on what they actually do, not on what a sales deck promises.
  • Defends Model Assets and Sensitive Data: Model weights, training data, system prompts, and embedded business logic have all become high-value targets. Knowing how attackers extract them is the only way to design controls that hold up under real pressure.
  • Anticipates Regulatory and Compliance Exposure: Agentic systems and MCP tool servers can route around organizational controls in ways that carry direct regulatory consequences, from sanctions screening to privacy obligations to financial reporting. Adversarial work surfaces with these exposures where they can be proactively fixed.
  • Connects Offense to Defensive Action: Every technique covered in this course traces back to a specific architectural decision, control, or detection opportunity. That mapping is what gives defenders something concrete to act on.

SEC536 puts these techniques in students' hands across two days of attacks against a realistic, production-shaped environment. By the end, participants can assess AI-integrated systems with confidence, communicate the findings to the executives funding the rollout, and reduce real exposure before attackers find the same gaps.

Completing SEC536 puts you in a small group of practitioners who can confidently assess AI systems under real attack conditions, a capability almost every organization deploying AI urgently needs and very few people currently have.

  • Stand Out in a Specialty with Few Experts: AI security is one of the fastest-emerging areas of cybersecurity, and qualified practitioners are in short supply. SEC536 places you at the front of that wave with hands-on experience attacking the systems organizations are actually building today.
  • Open Doors to AI-Focused Roles: The skills covered map directly to positions like AI Red Team Lead, AI Application Security Engineer, ML Platform Security Engineer, and AI Risk Specialist, as well as senior penetration testing roles where AI assessment is now part of the scope.
  • Carry Your Lab Environment Home: You leave with a takeaway VM containing every lab from the course. That makes the skills sticky, lets you keep practicing as new techniques emerge, and gives you a working reference environment you can use on real engagements.
  • Speak the Language of AI Risk to Leadership: SEC536 trains you to translate technical AI attacks into the business and regulatory framing that funds programs. Practitioners who can do that quickly become the trusted voice on AI risk inside their organizations.

Overall, SEC536 builds a rare and highly valued skill set, positions you for the AI security roles emerging across the industry, and gives you the practical experience to operate at the front edge of where cybersecurity is moving.

Course Schedule and Pricing

Looking for Group Purchasing Options?Contact Us
Showing 10 of 10

Benefits of Learning with SANS

Bryan Simon: Teacher Standing Next to Smartboard and Explaining Concept

Get feedback from the world’s best cybersecurity experts and instructors

OnDemand Mobile App

Choose how you want to learn - online, on demand, or at our live in-person training events

Close Up of Woman Holding a Pen and Documents

Get access to our range of industry-leading courses and resources