homepage
Open menu
Go one level top
  • Train and Certify
    Train and Certify

    Immediately apply the skills and techniques learned in SANS courses, ranges, and summits

    • Overview
    • Courses
      • Overview
      • Full Course List
      • By Focus Areas
        • Cloud Security
        • Cyber Defense
        • Cybersecurity and IT Essentials
        • DFIR
        • Industrial Control Systems
        • Offensive Operations
        • Management, Legal, and Audit
      • By Skill Levels
        • New to Cyber
        • Essentials
        • Advanced
        • Expert
      • Training Formats
        • OnDemand
        • In-Person
        • Live Online
      • Course Demos
    • Training Roadmaps
      • Skills Roadmap
      • Focus Area Job Roles
        • Cyber Defense Job Roles
        • Offensive Operations Job Roles
        • DFIR Job Roles
        • Cloud Job Roles
        • ICS Job Roles
        • Leadership Job Roles
      • NICE Framework
        • Security Provisionals
        • Operate and Maintain
        • Oversee and Govern
        • Protect and Defend
        • Analyze
        • Collect and Operate
        • Investigate
        • Industrial Control Systems
      • European Skills Framework
    • GIAC Certifications
    • Training Events & Summits
      • Events Overview
      • Event Locations
        • Asia
        • Australia & New Zealand
        • Latin America
        • Mainland Europe
        • Middle East & Africa
        • Scandinavia
        • United Kingdom & Ireland
        • United States & Canada
      • Summits
    • OnDemand
    • Get Started in Cyber
      • Overview
      • Degree and Certificate Programs
      • Scholarships
      • Cyber Aces
    • Cyber Ranges
  • Manage Your Team
    Manage Your Team

    Build a world-class cyber team with our workforce development programs

    • Overview
    • Why Work with SANS
    • Group Purchasing
    • Build Your Team
      • Team Development
      • Assessments
      • Private Training
      • Hire Cyber Professionals
      • By Industry
        • Health Care
        • Industrial Control Systems Security
        • Military
    • Leadership Training
  • Security Awareness
    Security Awareness

    Increase your staff’s cyber awareness, help them change their behaviors, and reduce your organizational risk

    • Overview
    • Products & Services
      • Security Awareness Training
        • EndUser Training
        • Phishing Platform
      • Specialized
        • Developer Training
        • ICS Engineer Training
        • NERC CIP Training
        • IT Administrator
      • Risk Assessments
        • Knowledge Assessment
        • Culture Assessment
        • Behavioral Risk Assessment
    • OUCH! Newsletter
    • Career Development
      • Overview
      • Training & Courses
      • Professional Credential
    • Blog
    • Partners
    • Reports & Case Studies
  • Resources
    Resources

    Enhance your skills with access to thousands of free resources, 150+ instructor-developed tools, and the latest cybersecurity news and analysis

    • Overview
    • Webcasts
    • Free Cybersecurity Events
      • Free Events Overview
      • Summits
      • Solutions Forums
      • Community Nights
    • Content
      • Newsletters
        • NewsBites
        • @RISK
        • OUCH! Newsletter
      • Blog
      • Podcasts
      • Summit Presentations
      • Posters & Cheat Sheets
    • Research
      • White Papers
      • Security Policies
    • Tools
    • Focus Areas
      • Cyber Defense
      • Cloud Security
      • Digital Forensics & Incident Response
      • Industrial Control Systems
      • Cyber Security Leadership
      • Offensive Operations
  • Get Involved
    Get Involved

    Help keep the cyber community one step ahead of threats. Join the SANS community or begin your journey of becoming a SANS Certified Instructor today.

    • Overview
    • Join the Community
    • Work Study
    • Teach for SANS
    • CISO Network
    • Partnerships
    • Sponsorship Opportunities
  • About
    About

    Learn more about how SANS empowers and educates current and future cybersecurity practitioners with knowledge and skills

    • SANS
      • Overview
      • Our Founder
      • Awards
    • Instructors
      • Our Instructors
      • Full Instructor List
    • Mission
      • Our Mission
      • Diversity
      • Scholarships
    • Contact
      • Contact Customer Service
      • Contact Sales
      • Press & Media Enquiries
    • Frequent Asked Questions
    • Customer Reviews
    • Press
    • Careers
  • Contact Sales
  • SANS Sites
    • GIAC Security Certifications
    • Internet Storm Center
    • SANS Technology Institute
    • Security Awareness Training
  • Search
  • Log In
  • Join
    • Account Dashboard
    • Log Out
  1. Home >
  2. Blog >
  3. How to Mature ICS Security with Metrics
370x370_Jason-D-Christopher.jpg
Jason D. Christopher

How to Mature ICS Security with Metrics

“If you’re not measuring, you’re just practicing.”

September 8, 2022

At SANS ICS Summit in 2019, I spoke about how to create an ICS security program. Having created several similar programs across multiple critical infrastructure organizations, I started with a simple premise: just start doing something. Most times, a simple starting point can just be counting something easy, such as how many facilities or systems are covered by your security policies. Other times, new projects can present opportunities to bake-in security metrics, like the percentage of the network with OT detection in place or easy quantification of other security controls. These starting points are just that—they won’t be perfect. Still, they can help tell a story that can unlock additional budget and resources or provide executive visibility to problems within the ICS security program that need to be solved.

In the past three years since that presentation, I have been continuing to help ICS security teams and their leaders better understand their security posture through measurement. The lessons learned over the years can now be found in ICS418: ICS Security Essentials for Managers, which I co-authored with fellow SANS ICS instructor Dean Parsons. Across several modules in section two of the course, we explore how to implement an OT security program with a qualified team of subject matter experts and capture meaningful metrics for not just guiding that team but for providing an overall strategy to the organization, including executive leadership.

There are many valid reasons why ICS security leaders should implement a metrics program. First and foremost, look at your engineering and operations counterparts. Their jobs (and in some cases, their very livelihood, when safety is involved) are deeply rooted in accurate and routine measurement. This was one of the reasons why, back in 2019, I told the audience at the SANS ICS Summit that I believe (and still do) that ICS/OT security will solve cyber security measurement before IT does. Why did I say that? Because we are surrounded by measurement. We live and breathe it daily. All we need to do is tap that energy and direct it toward cyber security concerns.

Other reasons for measurement are less about our culture and background and more about the practicality of using metrics. When working with ICS security leaders, I usually focus on a few key “whys” when it comes to creating metrics, including:

  • Increase Accountability: Data collection on specific metrics can identify personnel responsible for control implementation and provide input on effectiveness.
  • Improve ICS Security Effectiveness: Continued metric development and reporting will track trends for the ICS security program.
  • Demonstrate Compliance: Metrics can be built as compliance measures.
  • Provide Quantifiable Inputs for Resource Allocation Decisions: If built correctly, metrics can help identify weak points with quantifiable data that can support additional resources for the ICS security program.

These “whys” are adapted from the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-55, which is a great source document for anyone looking to start measuring items of interest in cyber security. While it’s IT-focused, there’s no need to recreate the wheel. ICS security practitioners should be well-versed in how to adapt documents like this for your own benefit.

Basic ICS/OT Security Metric Implementation Process

If the “whys” seem like something you want (accountability for security in the plant? Additional resources to address program gaps? Who doesn’t want that?), then the first question is usually, “what does it take to build a security metrics program?”

First, let’s breakdown the lifecycle for security metrics:

Picture1.png

Each step has specific considerations for team sizing, tools, and other resources. Let’s take a look at each step:

  1. Create (or Update) Metrics: The first step is always the hardest, so it’s important to understand what makes a metric useful. State your goals. Try to figure out what data is needed to build the metric. For example, if you want to examine how well your incident response and recovery program is performing, perhaps you’d leverage something like the following:Picture2.png

And—yes—update those metrics as needed! If a metric is not used after two cycles of review, drop it. If a targeted level of performance is achieved, move on. Let your leadership know as you implement a new metrics program that these will change over time as your ICS security posture improves.

2. Collect Data: Now that we have created (or updated) our metrics, how do we collect the data? Who actually has the data? ICS security requires multiple stakeholders, so the information may not be in your purview. As such, be prepared to ask other stakeholders for access to data. Further, this data may not all be digital (manual physical logs, for example, might need to be used for access control) and it may not all be objective. I’ve started more than a few metrics programs based on staff interviews; it’s not pretty, but it’s still valuable information.

3. Data Store: Unfortunately, many practitioners think about this after the fact and then end up with a series of Excel data sheets stored all over the place. But what if you’re planful about it? You can leverage not only searchable databases, but you can bake-in security provisions for the data with strict access controls. After all, this information will be a treasure trove of insight about your strengths and weaknesses. Consider data science storage techniques that can also aid in collection and the required analysis.

4. Analyze and Compile Data: With everything collected and properly stored, now you can look for linkages in the data (which can be challenging, depending on the data sources). Look to compile and aggregate the data in metrics as defined in step one. Did the data support your hypothesis? Does it tell a different story? Where required, leverage equations to provide quantifiable measurements and experiment with different charts, graphs, and visual tools to support further interpretation of the information being presented.

5. Report Metrics: Answer the question about what this data is telling you regarding your security program, again linking back to your original statements in step one above. This is where I’ve seen OT security leaders do great things by getting an engineer or data scientist “on loan” from another department (labs are FILLED with data science geeks that may also be interested in security). You may be limited on certain abilities here, but if you start with the original items in step one listed as examples you can always leverage simple spreadsheet charts and graphs.

6. (Actually) Use Metrics: Don’t just admire the problem, do something about it! Take the storytelling and move forward with a plan to address gaps, allocate budget, and execute on plans. The goal of using a metric is to trend toward improvement. There is no value in showing your team stagnating on a security issue, so be prepared to couple a metric with “why” it’s important and what to do about the gaps it shows. This will enable other leaders to help your cause and potentially address larger programmatic issues.

Sizing, Budgeting, and other Considerations for ICS Security Metrics

Ok, so admittedly that looks like a lot to do. And it’s not going to be easy, but this is something you can plan for and build over time. As such, there are a few “rules of the road” for anyone looking to begin this journey:

  • First (and this should be obvious), the cost of collecting and analyzing the data for a metric should not exceed the cost of implementing the security control in question.
  • Metrics should drive a change in behavior or an improvement in the ICS/OT security program. Be on the look out for abuse or poor morale and adjust your strategy accordingly.
  • Standardize where you can across other similar metrics programs. Is there already a workflow for engineers and operators to report metrics? Will it work for your efforts? Then adopt what you can and move forward!
  • Metrics collection, analysis, and reporting all cause a certain amount of overhead. You cannot just assign them to your team members without the potential of overwhelming them with additional work. In my experience, never assign more than two to four security metrics per full-time employee (FTE) on your security team (or be prepared for some burn out). Alternatively, establish a dedicated resource or team for the metrics effort.

For many of the organizations I’ve worked with, they will enter a pilot period where they attempt to measure something, maybe anything, that provides additional insights. After working with some champions and showing initial success, those teams will usually go on to expand the program with specific goals for measurement and can then create a workflow similar to what is captured above with dedicated tools and resources.

When to Start

Now.

Seriously, just start measuring something of interest. It doesn’t need to be precise, and it can just be an experiment. What data do you have access to? Where are your strengths and weaknesses, and how would you prove them? What chart or graph would you love to see about your OT security program? Don’t let perfect be the enemy of the good; start your metrics journey today, if you can.

This is just one of many topics we cover in ICS418, designed to help ICS/OT security leaders mature their program and their teams using practical tools based on real-world experience. The course cuts through the noise across dozens of industry standards, guidelines, and best practices to provide actionable guidance. Find out more and register for class at: www.sans.org/ics418

About the Author

Over the past 20 years, Jason D. Christopher has worked across multiple industries in unique roles ranging from engineering to incident response and national security. Most notably, Jason was the federal technical lead for the NERC CIPv5 while at the Federal Energy Regulatory Commission, where he was involved in several rulemakings and policy statements. Jason was also the program lead for the U.S. Department of Energy Cybersecurity Capability Maturity Model (C2M2). He has also served as a C-level executive, security researcher, and incident responder across his career. He is currently the Director of Cyber Risk for Dragos, Inc. focusing on ICS monitoring technology, threat intelligence, and industrial cyber risk. Jason has been invited to speak before the U.S. Congress on several occasions. He teaches ICS456: Essentials for NERC Critical Infrastructure Protection and is co-author of the upcoming course ICS418: ICS Security Essentials for Managers. Learn more about Jason here.

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Tags:
  • Industrial Control Systems Security

Related Content

Blog
ICS_Blog_Series-_A_Look_into_ICS_Cybersecurity_-_Social2.jpg
Industrial Control Systems Security
December 15, 2021
A Look Into ICS612: ICS Cybersecurity In-Depth: Part 1
ICS practitioners can immediately use their ICS612 training in real-world situations.
370x370_jeffrey-shearer.jpg
Jeffrey Shearer
read more
Blog
Untitled_design-43.png
Digital Forensics and Incident Response, Cybersecurity and IT Essentials, Industrial Control Systems Security, Purple Team, Open-Source Intelligence (OSINT), Penetration Testing and Red Teaming, Cyber Defense, Cloud Security, Security Management, Legal, and Audit
December 8, 2021
Good News: SANS Virtual Summits Will Remain FREE for the Community in 2022
They’re virtual. They’re global. They’re free.
370x370-person-placeholder.png
Emily Blades
read more
Blog
Industrial Control Systems Security
March 3, 2021
Industrial Control Systems Library
The SANS Industrial Control Systems Library is a central source for all ICS Security brochures detailing our courses, Posters, Surveys, Whitepapers, and our Defense Use Case papers. All of the assets below are .pdf downloads.Brochures2016: Deutsche ICS Brochüre2016: ICS Security Training...
SANS ICS
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters & Cheat Sheets
  • White Papers
  • Focus Areas
  • Cyber Defense
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • © 2023 SANS™ Institute
  • Privacy Policy
  • Contact
  • Careers
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn