Ted Gutierrez

Editor's Note: Today's guest blog is published by Ted Gutierrez, CISSP, GICSP, and GCIH, is the ICS & NERC CIP Product Manager at the SANS Institute and co-author of SANS ICS456 - Essentials for NERC CIP.

With the recent release of the E-ISAC and SANS ICS Defense Use Case (DUC) #5 which analyzed the cyber-attack that impacted Ukraine on December 23, 2015, I wondered how NERC CIP might have helped.  I want to preface this analysis with acknowledgement that the Ukrainian event was wholly contained at the distribution level of their electric system.  Except in very limited and specific situations, similar facilities in the United States would not have been under NERC regulation and therefore not subject to the CIP standards.  The Ukrainian event clearly shows us that while the higher voltage transmission systems are the backbone of an electric system, they don’t serve much practical good without the underlying distribution systems. NERC CIP gets a bad rap from many who claim that regulation doesn’t equal security.  I actually agree, but when it comes to CIP Version 5/6 it’s hard to make the case that CIP isn’t making a difference where applicable and when implemented in a holistic manner that balances the cyber security and compliance obligations.  Below, I’ll try to map the known Ukrainian events and the ICS Cyber Kill Chain to specific cyber security requirements in NERC CIP that might have helped if this had been an in-scope control center controlling in-scope substations.

Stage 1- Reconnaissance: This step is characterized by the attacker passively seeking information about the intended target and is perhaps the hardest to detect.  In this stage, attackers try to identify employees of interest, business locations, technologies in use, etc., in hopes of finding vulnerabilities.  CIP-011, Information Protection, is the CIP standard that would try to limit the disclosure of information that could lead to a compromise.  It’s unknown how much the attackers were aided by publicly available information but certainly any efforts to protect BES Cyber System Information would at least make this stage more difficult.

Stage 1- Targeting/Weponization: In the Ukrainian case, Microsoft Office documents were weaponized with embedded BlackEnergy 3 and were delivered to company employees with administrative or IT network responsibilities.  The specific assets used to access corporate email would likely be out of scope for NERC CIP (assuming proper network isolation and segmentation has taken place) but malicious code prevention measures such as AV, whitelisting, intrusion preventions systems, and monitoring included in CIP-007, System Security Management, might have detected something out of the ordinary.  The cyber security awareness and cyber security training requirements in CIP-004, Personnel & Training, might also have helped the recipients recognize the phishing attack and prevented it from succeeding.

Stage 1- C2: In this stage the attackers were able to establish a foothold and secure command and control capabilities.  But those weren’t needed long as they quickly obtained legitimate credentials harvested from directory servers on the corporate network giving them cloaked access to into the ICS network.  Again, depending on the configuration of the directory services these corporate systems would likely have been outside of the scope of NERC CIP.  But assuming that the SCADA environment would have existed within a CIP-005 required Electronic Security Perimeter (ESP), they would have had to have identified all inbound and outbound access permissions to the ESP.  Additionally, they would have had to use an Intermediate System which most certainly would have been in-scope for NERC CIP requiring all applicable CIP-007 and CIP-005 measures including multi-factor authentication for Interactive Remote Access.

Stage 2- ICS Specific Attack Development, Testing and Delivery:  In the Ukrainian case, the attackers planted a modified version of KillDisk software to specific systems selected because the unavailability of those systems after the actual attack would hinder recovery.   They also delivered custom malicious firmware updates to serial-to-Ethernet devices needed to communicate with field devices.  Whitelisting, active configuration management, security event monitoring, and alerting measures that could be implemented to meet various CIP-007 and CIP-010 requirements may have prevented (or at least detected) and provided early warnings of unauthorized software being installed.  Remote access to the operator HMIs provided attackers with direct ability to use the SCADA HMI software to impersonate an operator and initiate breaker open commands.  Implementation of the ports and services hardening requirements of CIP-007 may have identified remote access as an unneeded service requiring that it be disabled. In at least one location the attackers reconfigured a UPS to cause a localized power outage to further hamper recovery efforts.  This is one glaring gap in the NERC standards that I believe should be revisited as currently HVAC, UPS, and other support systems are not the focus of compliance monitoring, unless those systems are within an ESP.1 While not directly capable of impacting the Bulk Electric System their dependence on these secondary systems deserves some degree of protections.

Post-attack Response and Recovery:  A thorough Cyber Security Incident Response Plan with identified roles and responsibilities as required by CIP-008 may have helped in earlier identification of the incident.  Pure speculation on my part, but it’s possible that there were signs that might have been overlooked and a process for identification may have helped.  Likewise, a response plan that had been regularly tested as required in CIP-008 may have provided the opportunity to identify weaknesses or gaps in the plan. While not much has been written about the recovery efforts post-attack, it’s safe to assume that well documented recovery procedures and the availability of verified backup sources required by CIP-009 would have aided in restoration efforts.  A well-documented recovery plan that included availability of system spares or information for obtaining spares may have also helped in quicker replacement of the serial-to-Ethernet devices that were unrecoverable and rendered useless by the malicious firmware.

Beginning with the Beginning:  Accepted security dogma dictates that in order to protect your systems you must understand what systems you have and the role they play.  Application of a defined methodology similar to the bright-line criteria in CIP-002 would have helped to identify the systems critical to operating the Ukrainian electric system and therefore deserving of higher level of protections. And finally, the Security Management Controls in CIP-003 would have forced the thoughtful consideration and development of cyber security policies.  The requirements in the other standards that require plan development and review may have alerted the Ukrainian teams of weaknesses needing to be addressed.

I don’t pretend that NERC CIP is perfect - far from it.  But the fact that it is making a difference at least with the systems to which it is applicable is undeniable.  That also isn’t meant to suggest that a system subject to NERC CIP can’t be compromised - its effectiveness will always depend on specific design considerations and will be subject to human error, system malfunction, and attacker ingenuity.  With regard to Ukraine, if nothing else, had measures similar to those required by NERC CIP been implemented, they might have slowed the attackers down or tripped them into making a detectible misstep. By examining events like what happened in the Ukraine and understanding how NERC CIP may or may not have helped at each stage, we might identify what we need to do differently.  We can also have the conversations about whether or not we are protecting all the right assets.  While not required by regulation, there is nothing that precludes U.S. entities from implementing CIP-like cyber security measures in similar distribution networks.  The Ukrainian event should give distribution asset owners cause to consider adopting similar cyber security measures and I hope this analysis demonstrates that NERC CIP-like cyber security measures can make a difference.

1NERC Frequently Asked Questions - CIP Version 5 Standards; Version October 2015 

Bio: Ted Gutierrez, CISSP, GICSP, and GCIH, is the ICS & NERC CIP Product Manager at the SANS Institute and co-author of SANS ICS456 - Essentials for NERC CIP. Ted was most recently the Director of Operations Technology & NERC Compliance at Northern Indiana Public Service Company (NIPSCO) where he was responsible for compliance to NERC 693 and CIP standards and the support of the related operations technology systems. He has over twenty-five years of experience working in the electric utility, information technology, and manufacturing industries.