Abstract:
It’s no secret that artificial intelligence now plays a critical role in most modern enterprises. It does everything from screening resumes to triaging security alerts to even generating code, often without human oversight. We have, in essence, handed these models the keys to our kingdom, getting so consumed by the question of "can we" that we forgot to ask "should we."
Unfortunately, that question is quickly being answered for us by our adversaries. Every capability and advancement we’ve celebrated also hides a vulnerability we overlooked. The same model that screens a resume can be coerced into following an attacker’s hidden instructions. The same assistant that writes our code can be convinced to implement a hidden back door. The worst part is that organizations have no idea how prominent these attacks have become, or that they are exposing an attack surface that traditional penetration testing was never built to address. Prompt injection, jailbreaks, training-data poisoning, model extraction, and agentic exploitation are happening every day, and most security teams still lack any methodology to test for them.
This session, based on the new SEC536: AI Penetration Testing course, shifts focus from AI's capabilities to its role as a new attack vector, encouraging critical evaluation of its use. Live demonstrations will show how production-grade AI assistants can be manipulated to bypass safeguards, leak confidential data, and perform prohibited actions.
