AI for Security and Security for AI: Navigating Opportunities and Challenges

AI for Security and Security for AI: Navigating Opportunities and Challenges, published by SANS Institute in October 2025, examines generative AI security through three connected lenses: securing generative AI applications themselves, using generative AI to strengthen security operations, and defending against generative AI-powered threats. The paper draws on operational experience building and running cloud services, with a particular focus on how automated reasoning (formal verification) can supplement generative AI to catch errors that probabilistic models cannot catch on their own.

Key findings:

• Two thirds of organizations are now prioritizing AI within their security operations, according to IT services firm Capgemini
• 95% of US companies are using generative AI, per a Bain & Company survey, showing adoption has outpaced security readiness
• 66% of organizations expect AI to significantly impact cybersecurity in the coming year, but only 37% have a process to evaluate the security of AI systems before deployment, according to the World Economic Forum's Global Cybersecurity Outlook 2025
• Trained security engineers typically spend over an hour analyzing a single CVE; a contextually grounded generative AI-based CVE analyzer can cut that time from hours to seconds while keeping output consistent
• LLMs are functionally non-deterministic: identical inputs can produce different outputs, so security controls cannot rely on reproducible model behavior for critical operations
• LLMs process all inputs (system prompts, retrieved documents, tool outputs, user inputs) with equal privilege, meaning authorization and access control cannot be implemented inside the model itself
• LLMs are statistical pattern-completion engines rather than logical deduction engines, which is why they can generate convincing but factually incorrect outputs
• Automated reasoning (formal verification) can validate AI-generated code, infrastructure configurations, and authorization policies with mathematical certainty rather than statistical confidence
• Cedar, the open source authorization policy language behind Amazon Verified Permissions, is built using verification-guided development based on automated reasoning
• The Open Cybersecurity Schema Framework (OCSF) standardizes security data formats across tools, enabling generative AI to map security events to MITRE ATT&CK tactics, techniques, and procedures automatically
• Defense-in-depth for AI systems spans seven interconnected layers: policies/procedures/awareness, identity and access management, infrastructure protection, network and edge protection, application protection, data protection, and threat detection and incident response
• The paper's guidance holds that the higher the potential impact of an AI-driven security decision, the more human oversight it requires, citing Amazon's practice of never letting AI autonomously shut down critical services or make major production changes

The paper's throughline is that generative AI security cannot rely on the model as a trust boundary. Because LLMs are non-deterministic, treat all inputs with equal privilege, and generate plausible rather than verified outputs, security architecture has to sit outside the model, using external validation, automated reasoning, and layered defense rather than expecting the model to enforce its own guardrails. At the same time, the same properties that make generative AI risky to secure also make it a powerful accelerant for security operations, from CVE triage to SOC investigation, provided human oversight scales with the potential impact of the decision. The guidance draws on AWS's practical experience securing generative AI and agentic systems in production, and references established frameworks including the NIST AI Risk Management Framework, ISO/IEC 42001:2023, the EU AI Act, and the OWASP Gen AI Security Project.