Security Awareness Metrics

One of the challenges we have in security awareness is metrics, how do we measure the success of our awareness program? I believe this is one of the weakest areas of awareness (and so do others), and to be honest information security in general. A lot of people far smarter then me are trying to solve this problem, if you are interested in learning more about information security metrics in general, I highly recommend the securitmetrics.org maillist. Another great place to get started is Andrew Jaquith's book Security Metrics. One of my favorite insights is his criteria for good metrics.

1. Consistently measures (no subjective criteria).
2. Cheap to gather (preferably automated).
3. Expressed as a cardinal number or percentage.
4. Expressed using at least one unit of measure.
5. Contextually specific (i.e. relevant to decision makers so they can take action).

For security awareness, I feel there are two general categories for metrics. Categories that measure who took the training and metrics that measure the impact of the training.

• WHO: This measures how many people took the awareness training. Think metrics such as how many employees attended the security awareness workshop (often a sign in sheet) or how many employees took the online computer based training (often tracked by a SCORM compliant Learning Management System / LMS). These metrics are the more common metrics because they are simple. In addition, these metrics are required for compliance purposes (such as for PCI-DSS or ISO 27001). Unfortunately, these metrics do not tell you if your awareness program is making a difference.

• IMPACT: This measures how effective the training was, are you getting a return for your investment. This is by far the more difficult metric, but also the more important one. The challenge is finding good metrics that meet the requirements discussed above (requirement #2 is one of the hardest). Two good metrics that meet the requirements above are awareness surveys and awareness assessments. Awareness surveys are a great way to get a baseline set and they are simple to setup (if nothing else try my favorite www.surveymonkey.com.). These let you know things you may take for granted. For example, do your employees know they have a security team, do they know they are a target, do they know your basic policies? You can't enforce a policy they don't know about. I've included an example of an awareness survey. Awareness assessments replicate common human attacks, such as replicating a phishing attacks or scam. These are easy to automate (send out via email or SMS) then simply track who responds. If your awareness program is having an effect, the percentage of people who fall victim to these replicated assessments should be dropping.