SEC536: Adversarial AI - Penetration Testing AI Systems


More than 70% of cloud security incidents stem from customer misconfigurations or weak credential practices, according to Google's 2024 Threat Horizons report cited in the SANS research, not from vulnerabilities in the cloud platforms themselves.
Cloud IAM is dynamic, API-driven, and heavily automated, unlike the static, perimeter-based networks traditional IAM was designed for. Machine identities like service accounts and API tokens often outnumber human users by orders of magnitude and go unmanaged, and major cloud providers offer thousands of granular permissions that are difficult to interpret, leading to widespread over-permissioning.
Not well-suited to them. The research notes that many organizations continue deploying virtual firewall appliances in the cloud as a familiar crutch, but these tools become bottlenecks for east-west traffic in microservices architectures and rely on static rules and IP ranges that don't scale in ephemeral, cloud-native environments.
Two main reasons: a skills gap, since many security teams have deep traditional incident response experience but limited knowledge of cloud service logs, IAM misuse, and API-driven attacks, and a lack of telemetry, since cloud logs are fundamentally different from network-centric data, consisting instead of API calls, configuration changes, and identity events.
Cloud infrastructure entitlement management (CIEM) products and native IAM privilege/role analysis tools, such as Google's IAM Recommender, provide visibility into access risk, permission usage, and overprivileged identities, using analytics to identify unnecessary permissions and help enforce least privilege at scale.
The research frames data governance as moving toward "data as code," since training data ultimately determines what an LLM-powered chatbot or AI agent will do. Organizations that have not adapted their data governance practices for the cloud are described as likely to experience even more disruption when AI adoption exposes those same gaps.