SEC545: GenAI and LLM Application Security

The course starts with GenAI fundamentals, covering key concepts like Large Language Models (LLMs), embeddings, and Retrieval-Augmented Generation (RAG). Students will explore security risks unique to GenAI, including prompt injection, malicious models, and third-party supply chain vulnerabilities.

Section 2 dives into core components for GenAI apps, like vector databases, LangChain and AI agents. Students also explore deployment strategies, comparing cloud and on-premises setups with a focus on the security risks unique to each. The section concludes by introducing agents communication protocols such as MCP.

In Section 3, students continue exploring MCP security before diving into Transformers, the core technology behind LLMs. They examine the foundation of predictive modeling, evaluate secure hosting options for AI applications, and conclude with securing data orchestration pipelines and tools such as Airflow.

Section 4 focuses on MLOps and integrating security across pipelines. It covers model-specific attacks like serialization flaws and backdoors, then explores securing pipelines using controls such as model signing and automated scanning. The section ends with a hands-on AI threat modeling exercise using the MAESTRO framework.

Section 5 covers using AI for threat hunting and incident investigation and response, followed by a Capture the Flag (CTF) exercise. Students apply what they’ve learned to identify and remediate issues within AI infrastructure that includes Kubernetes, Docker Compose, MCP servers, Airflow, SageMaker, AWS Bedrock, and other cloud environments.