Effectively Benchmarking Your Phishing Awareness Program

The eBook Effectively Benchmarking Your Phishing Awareness Program, published by SANS Institute in December 2023, examines how organizations can benchmark phishing simulation results against external peers and internal business units. The eBook outlines the variables that affect benchmark validity, introduces the SANS Security Awareness Maturity Model® and the SANS Tiering Model, and explains why program maturity and simulation difficulty must be accounted for before comparing phishing metrics across organizations.

Key findings:

• Phishing simulations are generally measured using two core metrics: the undesired action rate (click rate) and the report rate
• The SANS Security Awareness Maturity Model® defines five stages of program maturity: Non-existent, Compliance-Focused, Promoting Awareness and Behavior Change, Long-term Sustainment and Culture Change, and Strategic Metrics Framework
• The SANS Tiering Model breaks phishing simulations into five tiers, ranging from Tier 1 (impersonal, misspelled, easily identifiable) to Tier 5 (highly personalized, cloned or spoofed emails targeting high-profile individuals)
• At least 11 variables affect the validity of a phishing benchmark, including workforce sample size, program history, simulation difficulty, and ease of reporting
• Tier 1 and Tier 2 simulations are considered appropriate for an entire workforce, while Tier 3 and Tier 4 simulations are typically reserved for specific roles or business units
• Tier 5 simulations, which target high-profile individuals with researched, highly personalized phishing, typically require separate assessment resources outside standard benchmark data
• Phishing benchmarking is a form of performance benchmarking, comparing metrics against similar organizations, direct competitors, or global peers regardless of industry
• Internal benchmarking compares phishing results across departments, business units, regions, and management levels within a single organization
• The most reliable external benchmark comes from sending an identical phishing simulation to comparable organizations with similar workforce demographics
• Relying too heavily on benchmark statistics instead of report and click rates can drive counterproductive behavior within a security awareness program

Across the eBook, the throughline is that comparing raw click and report rates between organizations is unreliable unless program maturity and simulation difficulty are normalized first. Tiering and maturity modeling give security teams a way to make "apples to apples" comparisons, both against external peers and across internal departments, rather than drawing conclusions from mismatched simulations. The guidance is based on SANS Institute's Security Awareness Maturity Model® and Tiering Model frameworks, applied to both external, cross-organization benchmarking and internal, department-level benchmarking of phishing simulation programs.