SEC536: Adversarial AI - Penetration Testing AI Systems


Experience SANS training through course previews.
Learn MoreLet us help.
Contact usBecome a member for instant access to our free resources.
Sign UpWe're here to help.
Contact UsKey findings:
The findings suggest that opaque-box interrogation is a viable way to reconstruct an AI agent's attack surface with no visibility into its source code, including tools introduced indirectly through MCP servers with no prompt-level documentation. At the same time, the tapering detection rates on larger tool sets point to a practical trade-off: low iteration limits reduce inference cost but risk incomplete attack surface discovery, meaning security teams need to tune iteration limits and run multiple interrogation passes when assessing agents with many available tools. Testing was conducted against three configurations of the open-source DVLA project (a LangChain-based vulnerable banking agent), one extended with Gmail tools through a Zapier MCP server, using OpenAI's GPT-4.1 to power the interrogator and 20 repeated interrogation runs per target to account for the non-deterministic behavior of LLMs.
An AI agent interrogator is a tool that probes an AI agent's natural language interface from the outside, with no access to its source code, to reconstruct the list of tools, functions, and capabilities it can invoke. Agent Interrogator, introduced in this SANS research, does this through a two-stage process: identifying high-level capabilities first, then enumerating the specific tools behind each one.
Yes. In testing, Agent Interrogator detected 100% of the tools made available through a Zapier MCP server connected to a test agent, even though the agent's hard-coded prompts gave the LLM no explicit documentation about those tools.
Across 20 repeated interrogation runs per test target, Agent Interrogator detected both functions on a baseline test agent in 95% of runs and reported zero hallucinated tools against a control agent configured with no tools at all.
Detection frequency dropped for tools discovered later in a session, from 100% down to as low as 45% for the last-discovered Gmail functions. The research attributes this to the framework's maximum iteration limit being too low to exhaustively work through a large tool set, not to a fundamental flaw in the interrogation approach.
Excessive agency describes an AI agent being granted capabilities, such as authentication functions, that should sit outside the agent's control with proper guardrails. The test agent used in this research includes a get_current_user function as a deliberate example of this vulnerability class.
Not fully yet. The current framework does not recursively interrogate a discovered tool if that tool is itself another AI agent with its own set of tools, which the research flags as a priority area for future work as agent-to-agent protocols like MCP and A2A become more common.



















