SEC411: AI Security Principles and Practices: GenAI and LLM Defense

Full Lab Details

• Lab 0: LLM Basics
• Level 0: Mission Briefing: Orient to the lab interface, toolbox, and keyboard shortcuts
• Level 1: Temperature & Creativity: Control output randomness from deterministic to creative
• Level 2: Context Windows & Tokens: Explore how tokenization and window size constrain model behavior
• Level 3: System Prompts & Roles: Shape model behavior through system prompts as a security boundary
• Level 4: Boundaries & Blind Spots: Probe knowledge cutoff, confabulation, safety refusals, and prompt fragility
• Lab 1: Tokenizer Security Explorer
• Level 1: Discovery: Analyze basic tokenization and its security impact
• Level 2: Model Comparison: Compare tokenization behaviors across multiple models
• Level 3: Attack & Defense: Hands-on exploration of tokenization attack techniques and token-aware defense implementation
• Level 4: Tokenizer Forensics: Fingerprint models and analyze tokenization patterns, including BPE vs. WordPiece comparison and multilingual filter bypass techniques
• Level 5: Advanced Challenges: Comprehensive tokenization attack/defend scenarios

Full Topic Details

• Security implications of LLM architecture, training processes, and inference mechanisms
• AI system attack surface mapping and threat modeling using MITRE ATLAS
• Tokenization vulnerabilities and exploitation techniques across different model families
• AI supply chain security, training data poisoning, and model manipulation tactics

Full Lab Details

• Lab 2: Prompt Injection Attacks and Defenses
• Level 1: System Prompt Attacks: Override weak system prompts to extract protected information
• Level 2: Input Filtering: Bypass keyword blacklist filters using encoding and obfuscation
• Level 3: Output Filtering: Extract sensitive data past post-LLM response pattern scanning
• Level 4: Semantic Detection: Defeat LLM-based intent classification through social engineering and payload splitting
• Level 5: Challenge Mode: Progressive defense challenges where each task isolates a single defense layer
• Lab 3: RAG Security Attacks and Defenses
• Level 1: Information Leakage: Exploit RAG knowledge bases through indirect queries
• Level 2: Semantic Bypass: Circumvent input filters using semantic manipulation
• Level 3: Document Poisoning: Inject malicious documents and craft trigger queries
• Level 4: Access Control Bypass: Escalate privileges from guest to captain-level access
• Level 5: Defense Implementation: Configure security controls for filtering, monitoring, and source citation

Full Topic Details

• Training pipeline security with data validation, poisoning detection, and secure infrastructure
• Inference runtime defenses using input sanitization, output validation, and semantic analysis
• OWASP Top 10 for LLM Applications, from prompt injection to model theft
• Advanced prompt attack techniques such as jailbreaking, system prompt extraction, and context manipulation
• Advanced prompt-injection techniques, including instruction manipulation, encoding and obfuscation, conversational-window exploitation, and social-engineering tactics
• RAG system security including vector database protection, retrieval poisoning, and access control
• Production security controls that balance protection and usability

Full Lab Details

• Lab 4: MCP Protocol Security
• Level 1: Reading the Fine Print: Discover hidden instructions in tool descriptions that poison LLM behavior
• Level 2: The Switcheroo: Detect rug pull attacks where servers mutate descriptions between connections
• Level 3: Cross-Fire: Observe cross-server contamination through shared context in multi-tool environments
• Level 4: Lockdown Protocol: Configure defenses including human in-the-loop, description pinning, and context isolation

Full Topic Details

• Secure AI deployment architectures and API protections for LLM services
• Authentication, authorization, and monitoring integrated with existing security operations
• Security for agentic systems, including permission models, tool validation, and agent control
• Reasoning model security, focusing on chain-of-thought protection and defending against reasoning manipulation
• Incident response for AI security events and development of enterprise AI security programs
• MCP protocol security, including attack surface and defensive controls
• Unified detection, monitoring, and incident-response practices for context-injection attacks, covering direct prompts, RAG/retrieval poisoning, tool and protocol abuse, and persistent agent-memory poisoning
• Current-events case study examining Anthropic's Fable 5 and Mythos 5 safety disclosures alongside a real US government access suspension, translating a fast-moving capability and governance story into concrete organizational defenses