In my last blog post we discussed the two different types of metrics for security awareness programs, metrics that measure compliance and metrics that measure impact. Most organizations focus on compliance metrics, such as how many employees took the training. These metrics are relatively easy. Today I would like to focus on measuring impact, are we reducing risk? I feel this is still a relatively young field and I'm hoping to develop some discussion and present some ideas on these metrics. I also feel impact metrics are important if we are going to demonstrate the value of security awareness.
I feel one of the best way to measure impact is replicate the same type of human attacks that we see today. By human attacks I mean attacks that exploit human vulnerabilities. The resent spat of LinkedIn emails (see image to left) are one example of attacks targeting humans. If a user is fooled into clicking on a link, their browser is sent to a malicious website which attempts to hack into their system. By sending out similar phishing emails to employees, you can measure how many employees click on them. If your awareness program is having an impact, the number of people clicking on such emails should go down over time. In addition, you can use these assessments to provide a feedback loop. If an employee falls victim to one of your phishing emails perhaps have them go to an internal website that explains they just fell victim to a test, and then have them take additional training on how to protect themselves.
If you think about it, email phishing assessments meet our standards for a good metric. The metric provides a specific quantitative value. You send so many emails out, you then measure how many opened the email and then how many actually clicked on the link (one interesting data point I'm starting to see is that a small percentage may open these emails, but a large percentage that open the email click on the link). The metric is very easy to reproduce, you simply send out emails that can be tracked. This is something you can automate and do weekly or even daily. Finally, you have valuable information that you can act upon. If too many people are clicking on phishing related attacks, you need more training for the end user on these types of attacks.
What other examples of experiences do you have with metrics that track the impact of your awareness program?