SEC536: Adversarial AI - Penetration Testing AI Systems


Experience SANS training through course previews.
Learn MoreLet us help.
Contact usBecome a member for instant access to our free resources.
Sign UpWe're here to help.
Contact UsKey findings:
The survey's core insight is that no single testing method covers every application type or vulnerability class. Manual penetration testing consistently outperforms automated tools because human testers understand intended application logic well enough to find business-logic and access-control flaws that scanners miss, while SAST, DAST, and SCA earn their place through ease of integration and lower cost rather than raw detection value. The recommended approach is to layer methods across the application lifecycle rather than rely on any one technique. Respondents were security administrators/analysts, security architects, product security professionals, and application developers, drawn from organizations of varying sizes, with roughly 80% confirming their organization develops and supports applications or APIs. Headquarters were concentrated in the United States and Europe, though more than 20% of respondent organizations had operations across nearly every global region.
There's no single answer across all application types: manual penetration testing scores highest for REST APIs, native mobile apps, and single-page web apps, while DAST scores highest for traditional form-based web applications, according to the SANS Application and API Security Survey 2024.
Software Composition Analysis (SCA) and Static Application Security Testing (SAST) are rated as the easiest testing techniques for development teams and the business to accept, while bug bounty programs and threat modeling rank as the hardest to accept.
54% of organizations have implemented runtime security protection (such as RASP) for at least one of their applications, and 53% have done so for at least one API.
35% of organizations report that more than half of their applications or APIs are built as microservices, while only 11% report having less than 10% built this way, suggesting a broader shift away from monolithic application design.
Manual penetration testing benefits from pre-validated findings and human testers' ability to understand intended application logic, allowing them to identify business-logic flaws and broken access controls that automated tools often miss or falsely flag.
Matt Bromiley is a Lead Solutions Engineer at LimaCharlie and SANS Certified Instructor. He serves as a GIAC Advisory Board member, a SME for the SANS Security Awareness, and a technical writer for the SANS Analyst Program.
Learn more

David has 20+ years of experience in vulnerability management, application security, and DevOps. He's developed technical security training initiatives, and believes vulnerability management is one of the most important foundations of cybersecurity.
Learn more


















