Group Purchasing
Group Purchasing
AI-FOCUSEDNEW

SEC545: GenAI and LLM Application Security

SEC545Cloud Security, Artificial Intelligence
  • 5 Days (Instructor-Led)
  • 30 Hours (Self-Paced)
Course authored by:
Ahmed AbuGharbia
Ahmed AbuGharbia
SEC545: GenAI and LLM Application Security
Course authored by:
Ahmed AbuGharbia
Ahmed AbuGharbia
  • GIAC AI Platform Security (GAIPS)
  • 30 CPEs

    Apply your credits to renew your certifications

  • In-Person, Virtual or Self-Paced

    Attend a live, instructor-led class at a location near you or remotely, or train on your time over 4 months

  • Advanced Skill Level

    Course material is geared for cyber security professionals with hands-on experience

  • 20 Hands-On Lab(s)

    Apply what you learn with hands-on exercises and labs

Learn to protect AI applications end to end through hands-on training in LLM security, RAG defense, agent security, and MLOps protection.

Course Overview

SEC545 teaches defenders how to secure GenAI and LLM applications in production. The course focuses on how to protect AI systems by threat modeling GenAI architectures, defending RAG and vector database pipelines, mitigating prompt injection, and hardening MLOps workflows across cloud, hybrid, and local environments.

SEC545 aligns to the SANS Secure AI Blueprint’s Protect AI pillar and is designed for practitioners responsible for securing AI-enabled applications end to end.

Hands-On GenAI and LLM Application Security Training

With the anticipated transformative impact of Generative AI (GenAI) on industries and technologies, the need for robust security practices to address its risks has never been more critical. SEC545 training equips students with the necessary knowledge to secure GenAI applications.

SEC545 training offers a comprehensive exploration of GenAI technologies, starting with foundational principles and underlying frameworks. Students rigorously evaluate security risks by identifying and analyzing real-world threats affecting GenAI applications and will progressively learn to implement security best practices by exploring strategies to safeguard GenAI systems effectively.

By the end of this training, students will possess a holistic understanding of GenAI security, empowering them to design, deploy, and defend GenAI systems in a rapidly evolving technological landscape.

The field of Generative AI is advancing rapidly, often outpacing the development of established security practices. SEC545 training provides an in-depth exploration of GenAI technologies, starting with core principles and underlying technologies. Students will assess security risks by identifying and analyzing real-world threats impacting GenAI applications. As students progress, they will learn to establish security best practices by exploring different measures for securing GenAI applications effectively. This course aims to shape effective GenAI security approaches by supporting the security community through ongoing research and practical insights.

The course begins by introducing students to the fundamentals of GenAI, covering key concepts and terminologies such as Large Language Models (LLMs), embeddings, and Retrieval-Augmented Generation (RAG). Students examine the security risks associated with GenAI, including prompt injection attacks, malicious models, and third-party supply chain vulnerabilities, following this by diving into the essential components needed to build a GenAI application, including coverage of vector databases, LangChain, AI agents, and MCP. The course then progresses to focus on hosting options for GenAI applications, discussing options for local deployment, cloud solutions, and platforms like AWS Bedrock.

The course then shifts focus to MLOps, covering data flow orchestration, data pipelines, and operational tasks for training and deploying models. Students explore tools such as Airflow and learn how to apply security throughout the AI deployment process, including hands-on exercises in training models with AWS SageMaker, defending against model serialization attacks, securing AI deployment pipelines, and threat modeling using MAESTRO.

The class concludes with an overview of using AI for security use cases, including incident handling. Students then participate in a Capture the Flag (CTF) exercise, applying what they have learned to identify vulnerabilities and implement fixes in the AI infrastructure they built over the previous four days.

Author Statement

“Emerging technologies often bring substantial value, transforming industries and opening new possibilities. However, their rapid adoption also introduces complex risks that are frequently not fully understood at the outset. As these technologies evolve, the nature and scale of associated risks can shift in unexpected ways, making it challenging to anticipate their full impact. This pattern has been clear with technologies like cloud computing, where the pace of innovation often surpasses our understanding of its security implications. The greater the potential of a technology, the more complex its associated risks.

AI, particularly generative AI, represents the next major wave of transformation, with the potential to reshape nearly every application. Organizations are increasingly focusing on the full lifecycle of AI applications, including data pipelines, model training, deployment, and overall MLOps. As a result, security leaders must expand their focus beyond traditional application and system security to also encompass these operational areas, ensuring that the entire AI workflow is protected.

This course aims to deepen students’ understanding of GenAI and its security challenges, equipping them with the skills to proactively manage and mitigate these risks. As the industry evolves, so will this course, ensuring that our approach to securing GenAI applications, infrastructure and operations remains at the forefront.”

- Ahmed Abugharbia

What You'll Learn

  • Understand how to protect AI systems by identifying security risks in GenAI, LLM, and agentic AI application architectures
  • Explore model fine-tuning, customization, and secure deployment strategies
  • Identify GenAI-specific threats, vulnerabilities, and practical mitigation techniques
  • Secure RAG pipelines, vector databases, and model-serving workflows to protect AI data flows from leakage, poisoning, and abuse
  • Apply defensive controls across MLOps environments and AI-enabled application stacks in support of a Protect AI strategy
  • Perform AI threat modeling using the MAESTRO framework to identify and prioritize AI-specific risks
  • Align Protect AI security practices with existing cloud, DevSecOps, and enterprise security frameworks

Business Takeaways

  • Understand how modern GenAI and agentic AI applications are designed, deployed, and secured
  • Identify business risks introduced by foundation models, AI tools, agents, and MLOps workflows
  • Protect AI infrastructure by mitigating security threats across models, data pipelines, vector stores, and supporting services
  • Implement end-to-end AI security controls from development through production and continuous operations
  • Integrate Protect AI principles with existing cloud security, governance, and compliance frameworks
  • Enable responsible AI adoption while maintaining business agility, resilience, and trust

Course Syllabus

Explore the course syllabus below to view the full range of topics covered in SEC545: GenAI and LLM Application Security.

Section 1GenAI, Large Language Models (LLMs), and Security Risks

The course starts with GenAI fundamentals, covering key concepts like Large Language Models (LLMs), embeddings, and Retrieval-Augmented Generation (RAG). Students will explore security risks unique to GenAI, including prompt injection, malicious models, and third-party supply chain vulnerabilities.

Topics covered

  • GenAI Introduction and Concepts
  • Fine-Tuning Models
  • Augmenting GenAI Knowledge
  • Safe Use and Moderation

Labs

  • LLMs and Prompt Injection
  • Fine-tuning OpenAI Models
  • Compromising Vector Database
  • Safe Use and Moderation

Full Topic Details

  • GenAI Introduction and Concepts
    • General AI and Generative AI
    • Large Language Models (LLMs)
    • Retrieval-Augmented Generation (RAG)
    • GenAI Application Components Security
    • Prompt Injection
  • Fine-Tuning Models:
    • OpenAI fine-tuning
    • File-tuning risks and models’ access
  • Augmenting GenAI Knowledge
    • Vector Databases
    • Knowledge Sources
    • Poisoning Data Sources
    • Prompt and instruction Poisoning

Section 2Securing GenAI Applications

Section 2 dives into core components for GenAI apps, like vector databases, LangChain and AI agents. Students also explore deployment strategies, comparing cloud and on-premises setups with a focus on the security risks unique to each. The section concludes by introducing agents communication protocols such as MCP.

Topics covered

  • AI Agents
  • GenAI Applications Architecture
  • AI Development Frameworks Security
  • Agents Communication Protocols

Labs

  • Pivoting from LLMs
  • Compromising LLM Supply Chain
  • Langchain Security
  • Model Context Protocol (MCP)

Full Topic Details

  • AI Agents
    • Agents’ design and capabilities
    • MCP Servers
    • Agents’ security risks
  • GenAI applications Architecture
    • GenAI application components
    • LLM Providers
    • Supply chain issues
  • AI Development Frameworks Security
    • Langchain components and architecture
    • Reasoning and chain of thoughts
    • Langchain security vulnerabilities
  • Agents Communication Protocols
    • MCP, A2A, and ACP
    • MCP Security Issues
    • Agents Authentication and Authorizations

Section 3Agentic AI Security

In Section 3, students continue exploring MCP security before diving into Transformers, the core technology behind LLMs. They examine the foundation of predictive modeling, evaluate secure hosting options for AI applications, and conclude with securing data orchestration pipelines and tools such as Airflow.

Topics covered

  • MCP Attacks and OAuth Security
  • Transformer Architecture Fundamentals
  • Hosting GenAI applications
  • Data Workflow Orchestration

Labs

  • Attacking MCP Infrastructure 1
  • MLSecOps – Securing AI Deployment Pipeline
  • Attacking MCP Infrastructure 2
  • AWS Bedrock
  • Attacking Airflow

Full Topic Details

  • MCP Attacks and OAuth Security
    • Client/Server Authentication and Authorization
    • OAuth and MCP
    • MCP Attacks such as leaking Auth tokens, Memory Poisoning and Pull The Rug
  • Transformer Architecture Fundamentals
    • Mathematical Foundations of Predictive Modeling
    • Deep dive on embeddings and vectors
    • Transformers design and Architecture
  • Hosting GenAI applications
    • Hosting options: On-prem, Cloud, Hybrid
    • Cloud AI services such as AWS Bedrock
    • Huggingface and the security risks
    • Hosting models locally and security considerations
  • Data Workflow Orchestration
    • Data Workflow Orchestration tools and infrastructure
    • Security risks and mitigation techniques
    • Securing Airflow

Section 4MLSecOps and Securing GenAI Applications Lifecycle

Section 4 focuses on MLOps and integrating security across pipelines. It covers model-specific attacks like serialization flaws and backdoors, then explores securing pipelines using controls such as model signing and automated scanning. The section ends with a hands-on AI threat modeling exercise using the MAESTRO framework.

Topics covered

  • Machine Learning Ops (MLOps)
  • Hosting Models
  • MLSecOps
  • AI Threat Modeling

Labs

  • Training Model Using SageMaker
  • Model Serialization Attacks
  • MLSecOps - Securing AI Deployment Pipeline
  • Threat Modeling with MAESTRO

Full Topic Details

  • Machine Learning Ops (MLOps)
    • MLOps technologies and frameworks
    • MLOps processes: Data processing, Model training, deployments and security considerations
    • AWS Sagemaker
  • Hosting Models
    • Huggingface Transformers Library
    • Fine-tuning models for security
    • Pickle format, security vulnerabilities, and scanning models
  • MLSecOps
    • Applying security controls into ML Pipelines
    • Signing models
    • Scanning Models
  • AI Threat Modeling
    • Threat Modeling frameworks
    • OWASP ASI Threat Taxonomy
    • MAESTRO Framework

Section 5AI for Security

Section 5 covers using AI for threat hunting and incident investigation and response, followed by a Capture the Flag (CTF) exercise. Students apply what they’ve learned to identify and remediate issues within AI infrastructure that includes Kubernetes, Docker Compose, MCP servers, Airflow, SageMaker, AWS Bedrock, and other cloud environments.

Topics covered

  • Incident handling and Investigation with AI

Labs

  • Investigating Incidents Using Investigator MCP Server
  • CTF

Full Topics Details

  • Incident handling and Investigation with AI
    • Incident Handling
    • Incident handling tools
    • Investigator MCP Server

Things You Need To Know

Cloud Accounts

SANS provides students with time-limited AWS accounts 24 hours before class starts for completing the labs. Students can log in to their SANS account and visit the MyLabs page to download their cloud credentials the day before class begins.

Mandatory Laptop Requirement

Students must bring their own system configured according to these instructions.

A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will likely leave the class unsatisfied because you will not be able to participate in hands-on exercises that are essential to this course. Therefore, we strongly urge you to arrive with a system meeting all the requirements specified for the course.

Students must be in full control of their system's network configuration. The system will need to communicate with the cloud-hosted student VM using a combination of HTTPS, SSH, and SOCKS5 traffic on non-standard ports. Running VPN, intercepting proxy, or egress firewall filters may cause connection issues communicating with the student VM. Students must be able to configure or disable these services to connect to the lab environment.

Bring Your Own Laptop Configured Using the Following Directions

A properly configured system is required for each student participating in this course. Before starting your course, carefully read and follow these instructions exactly:

  • Host Operating System: Latest version of Windows 10, macOS 10.15.x or later, or Linux that also can install and run the Firefox browser described below.
  • Fully update your host operating system prior to the class to ensure you have the right drivers and patches installed.

Mandatory Host Hardware Requirements

  • CPU: 64-bit 2.5+ GHz multi-core processor or higher
  • Wireless Ethernet 802.11 B/G/N/AC
  • Local Administrator Access within your host operating system
  • Must have the ability to install Firefox, enable a Firefox extension, and install a new trusted root certificate on the machine.

Mandatory Software Requirements

Summary

Before beginning the course you should:

After you have completed those steps, access the SANS provided AWS account to connect to the SANS Cloud Security Flight Simulator and connect to the SEC545 student VM. The SEC545 Instance hosts an electronic workbook, VSCode, Gitlab, and Terminal services that can be accessed through the Firefox browser.

Your course materials include a "Setup Instructions" document that details important steps you must take before you travel to a live class event or start an online class. It may take 30 minutes or more to complete these instructions.

Your class uses an electronic workbook for its lab instructions. In this new environment, a second monitor and/or a tablet device can be useful for keeping class materials visible while you are working on your course's labs.

If you have additional questions about the laptop specifications, please contact customer service.

SEC545 is designed for practitioners in Protect AI roles who are responsible for securing GenAI and LLM-powered applications, AI-enabled workflows, and the infrastructure that supports them. This includes

  • AI/ML Security Engineers
  • AI Security Specialists
  • Application Security Engineers working with GenAI or LLM applications
  • MLSecOps Engineers
  • AI SOC Orchestrators
  • Blue team leads for AI environments
  • Cloud Security Engineers responsible for GenAI deployments.

Many of these titles are still being defined inside organizations. SEC545 gives you the practical skills to step into them with credibility as the roles form, rather than waiting for hiring patterns to catch up.

The GIAC AI Platform Security (GAIPS) certification validates a practitioner’s ability to audit and secure Generative AI applications and large language model (LLM) development pipelines. GAIPS certification holders demonstrate hands-on skills protecting data flows, model integrations, APIs, and deployment workflows against emerging threats across the AI lifecycle.

  • Generative AI common terminology and fundamental concepts 
  • Tuning, augmenting, and aligning foundational models for custom business requirements 
  • Designing and securing agentic systems and AI integrations 
  • AI application architecture and development frameworks 
  • Implementing MLOps and MLSecOps practices 
  • Responsible AI usage and risk management strategies

More Certification Details

  • Electronic and printed courseware
  • Mp3 audio files of course lecture
  • SANS provisioned AWS account
  • SANS provided OpenAI API Key

Students should have working familiarity with the following technologies and tools before attending SEC545:

  • Linux command line basics
  • AI coding assistants such as ChatGPT and Claude Code
  • Version control with Git and CI/CD pipelines (GitLab CI)
  • Code editors, particularly VSCode with AI extensions
  • Container fundamentals and introductory Kubernetes
  • AWS fundamentals

Preparing for SEC545

SEC545 makes heavy use of DevOps, MLOps, and cloud-native tooling throughout the labs. Students who arrive with hands-on familiarity with the following will get more out of the lab time:

While many GenAI systems run in cloud environments, SEC545 is a defensive AI course focused on protecting the AI application stack itself, not a general cloud infrastructure security course.

Depending on your current or desired future role, the courses below are great next steps in your cybersecurity journey.

SEC545: GenAI and LLM Application Security aligns with the Protect AI pillar by teaching defenders how to secure GenAI and LLM applications in production. The course focuses on protecting AI systems by threat modeling GenAI architectures, defending RAG and vector database pipelines, mitigating prompt injection, and hardening MLOps deployments across cloud and on-prem environments.

While many GenAI systems run in cloud environments, SEC545 is a defensive AI course focused on protecting the AI application stack itself, not a general cloud infrastructure security course. Students learn to identify security risks in GenAI and LLM application architectures, secure RAG pipelines, vector databases, and model-serving workflows to protect AI data flows from leakage, poisoning, and abuse, and apply defensive controls to MLOps environments and AI-enabled application stacks in support of a Protect AI strategy.

GenAI and Large Language Model (LLM) application security refers to the strategies, tools, and practices used to protect generative AI systems from misuse, manipulation, and cyber threats. As organizations increasingly build and deploy GenAI-powered applications, these systems become valuable targets for attackers. Unlike traditional software, GenAI introduces unique risks, including prompt injection attacks, malicious model outputs, and vulnerabilities in third-party tools or models. Without strong security measures, GenAI applications can inadvertently leak sensitive data, generate harmful content, or expose organizations to supply chain attacks.

Prioritizing security in GenAI applications ensures that these powerful technologies can be used safely, ethically, and reliably. Securing GenAI is essential not only for protecting business assets but also for maintaining public trust.

This 5-day course provides a comprehensive look at GenAI and LLM application security, covering everything from prompt-level risks and supply chain vulnerabilities to in-depth topics like model training, MLSecOps, agent infrastructure, and cloud deployment scenarios. Students gain hands-on experience in threat modeling real-world GenAI pipelines and defending AI infrastructure that includes components such as Airflow, SageMaker, AWS Bedrock, and containerized environments.

Why it matters:

  • Prevent data leaks from malicious prompts or compromised models
  • Reduce risk of manipulation through prompt injection or model abuse
  • Protect third-party supply chains involved in building GenAI apps
  • Secure full GenAI stacks, including agents, pipelines, and infrastructure
  • Ensure regulatory compliance and responsible AI use
  • Safeguard organizational reputation by preventing harmful or biased outputs

SEC545 will strengthen your cybersecurity career by equipping you with the skills needed to secure one of the fastest-growing areas in technology: Generative AI. As more organizations adopt GenAI tools, there is a growing demand for security professionals who can identify risks, implement controls, and build secure AI applications. This course provides both technical knowledge and strategic guidance, helping you stand out in a competitive job market.

The 5-day course goes significantly further than its 3-day predecessor. It includes new labs on threat modeling with MAESTRO, securing MLOps pipelines, training models with SageMaker, and attacking agent infrastructure through MCP. Students gain hands-on experience defending against advanced GenAI threats across a real deployment pipeline. This level of depth gives practitioners a unique edge in applying GenAI security concepts in production environments.

Career benefits:

  • Boost your value with specialized GenAI security skills
  • Demonstrate proficiency with modern AI infrastructure like LangChain, Bedrock, and MCP
  • Stay ahead of emerging security trends in LLMs, agents, and AI-driven apps
  • Build hands-on experience with threat modeling, MLSecOps, and cloud AI security
  • Strengthen leadership potential by integrating GenAI into enterprise security strategies
  • Align AI security with organizational risk management and compliance frameworks

Relevant Job Roles

Cloud Security Engineer Training, Salary, and Career Path

Cloud Security

Cloud Security Engineers integrate advanced security measures into cloud and cloud-native environments, maximize security automation within DevOps workflows, and proactively mitigate threats to safeguard modern cloud infrastructures.

Explore learning path

Cloud Security Analyst Training, Salary, and Career Path

Cloud Security

A Cloud Security Analyst monitors and analyzes activity across cloud environments, proactively detects and assesses threats, and implements preventive controls and targeted defenses to protect critical business systems and data.

Explore learning path

Cybersecurity Architecture (OPM 652)

NICE: Design and Development

Responsible for ensuring that security requirements are adequately addressed in all aspects of enterprise architecture, including reference models, segment and solution architectures, and the resulting systems that protect and support organizational mission and business processes.

Explore learning path

Technology Research and Development (OPM 661)

NICE: Design and Development

Responsible for conducting software and systems engineering and software systems research to develop new capabilities with fully integrated cybersecurity. Conducts comprehensive technology research to evaluate potential vulnerabilities in cyberspace systems.

Explore learning path

Network Operations (OPM 441)

NICE: Implementation and Operation

Responsible for planning, implementing, and operating network services and systems, including hardware and virtual environments.

Explore learning path

Software Security Assessment (OPM 622)

NICE: Design and Development

Responsible for analyzing the security of new or existing computer applications, software, or specialized utility programs and delivering actionable results.

Explore learning path

Enterprise Architecture (OPM 651)

NICE: Design and Development

Responsible for developing and maintaining business, systems, and information processes to support enterprise mission needs. Develops technology rules and requirements that describe baseline and target architectures.

Explore learning path

Secure Systems Development (OPM 631)

NICE: Design and Development

Responsible for the secure design, development, and testing of systems and the evaluation of system security throughout the systems development life cycle.

Explore learning path

Course Schedule and Pricing

Have Questions?Contact Us
Showing 10 of 23

Benefits of Learning with SANS

Bryan Simon: Teacher Standing Next to Smartboard and Explaining Concept

Get feedback from the world’s best cybersecurity experts and instructors

OnDemand Mobile App

Choose how you want to learn - online, on demand, or at our live in-person training events

Close Up of Woman Holding a Pen and Documents

Get access to our range of industry-leading courses and resources