Recently I was asked a good question by Michael Allen that made me think. Specifically he asked "What would be the best methods/approach to measure the effectiveness of our awareness program"? After noodling on this for a bit my answer is it depends on your organization, what are you attempting to achieve for your organization? Every organization has their own unique goals, and that is what you will want to measure. Some examples include
- Perhaps your organization is interested in reducing security costs. In one organization I worked with they were able to save the costs of half a FTE (Full Time Employee) by simply reducing the number of infected systems through aware employees. This cost savings not only paid for the awareness program, but freed up resources for other security related work (not to mention the benefits the awareness program provided on numerous other topics). So here the metric was cost.
- Perhaps your organization is concerned about reducing risk. In this case you need to identify the top risks to your organization and measure the behaviors that reduce those risks. For example, if you feel email is the top human attack vector to your organization then that is what you want to measure, perhaps with phishing assessments. I have found phishing assessments to be very effective. If the physical security of desktops and offices are important, then you can do nightly sweeps, checking all the offices and computers in your organization. If you are attempting to measure a risk that is hard to measure through behaviors, such as password reuse or Cloud use, then a survey may be another good approach. Once again I have had very good results using surveys.
- Perhaps your organization is concerned about compliance, then you will want to track who has successfully completed your training, or how you are communicating your training.
Ultimately the best metrics depende on your goals, every organization is different. This is why we have created the Metrics Matrix resource, a spreadsheet with over twenty different metrics you can use to measure the effectiveness of your awareness program. More in the Security Awareness Planning Package.
