Defending Against OT Ransomware in Industrial Control Systems

Ransomware attacks against organizations are now one of the primary methods criminal and adversarial groups leverage to bring organizations, cities, and governments to their knees. The method and reasons behind the attacks vary. Still, the impact is often the same: potentially sensitive data is exfiltrated for financial or intellectual gain, and IT systems are encrypted and rendered unusable. Industrial Control System/Operational Technology (ICS/OT) servers and workstations are not immune to these styles of attacks as they use operating systems containing similar vulnerabilities to their IT counterparts but are often significantly lacking in patches or other forms of mitigations.

Critical Infrastructure Sectors

The US Cybersecurity and Infrastructure Security Agency (CISA) lists 16 Critical Infrastructure Sectors of primary concern for national economic security or public health and safety. Listed among these is the Critical Manufacturing sector, which comprises most of the goods we enjoy, from the clothes we wear and cars we drive to the materials used to build our homes. This sector, which we often take for granted but rely on for our daily existence, is often overlooked from a cybersecurity perspective. The Dragos 2022 Year in Review report revealed that 72% of Dragos tracked ransomware incidents targeted the manufacturing sector.

Figure 1: Dragos 2022 YIR Tracked Ransomware Attacks

Ransomware trends are increasing every year, and we can expect to see further attacks against the manufacturing sectors and other ICS/OT areas. However, all hope is not lost, as asset owners can take a defensible position and be ready when their plant moves into the adversary's crosshairs. To defend against these attacks, owners and operators must understand how to:

1. Respond to an attack,
2. Design and deploy defensible architecture,
3. Monitor networks,
4. Secure remote access connections, and
5. Ensure key vulnerabilities are remediated.

These five controls are detailed further in Figure 2 below and in the Five ICS Cybersecurity Critical Controls White Paper and subsequent Webcasts.

Figure 2: SANS Five Critical Cybersecurity Controls

In SANS ICS612: ICS Cybersecurity In-Depth, we train you and your organization's workforce to consider the ICS/OT environment from the ground up, leveraging many of the elements discussed in the Five Critical Cybersecurity Controls white paper while taking a more nuanced approach to certain critical topics. The course focuses on understanding the physical machine, or the process, first considering how it operates and developing an understanding of how the controllers, servers, workstations, and other systems interact with the plant floor. This information is critical to building defensible Programmable Logic Controller (PLC) programs, architectures, remote access solutions, network monitoring, and backup/restore capabilities, for example. As the course author Jason Dely often stresses, adversaries don't attack PLCs; they leverage the PLCs to attack the physical process. Thus, understanding how the machines and processes work is the first step to securing the ICS/OT environment.

Defensible Architecture

Circling back to ransomware, we often find that it and other forms of malware enter the ICS/OT environment via the IT/OT interconnect, vendor or third-party connections, or portable devices (also known as transient cyber assets). These points of entry require both technical and administrative controls to be put in place to protect from and detect against malicious behaviors. These common entry points into the ICS/OT environment highlight the importance of the fourth critical cybersecurity control, defensible architecture. In ICS612, we provide labs that help you understand data flows and recommend ways to architect the environment from the IT/OT boundary, establishing a Demilitarized Zone (DMZ) and then understanding how the manufacturing lines function and communicate between other lines, as shown in Figure 3.

Figure 3: Understanding and Securing Data Flows

Firewalls are a common security control leveraged in OT to enforce zones and conduits in an environment to restrict IT/OT traffic. They perform the necessary function of inspecting traffic across the OSI layers but often fail to understand OT protocols at the application layer. They also introduce points of failure and latency in the communication pathway. All these aspects are covered in lectures and hands-on labs in the course. For instance, PLC-to-PLC communication in manufacturing often requires very high-speed communication, and latency introduced by placing a firewall between the PLCs can cause operational disruptions, especially in high-speed conveyor and robotics applications. As students also find out in an adversary in the middle attack lab, some protocols, such as the Common Industrial Protocol (CIP) leveraged by Rockwell Controllers, are almost impossible to create Application Layer restrictions for as the data blob changes upon transaction. Commonly known service object fields can be parsed and understood, but the data elements themselves cannot.

Securing the OT edge is a crucial area the course emphasizes. It is vital when considering adversarial attack paths and ransomware placement. However, the dependency between IT and OT systems is significant, especially in manufacturing, where Manufacturing Execution Systems (MESs) and Enterprise Resource Planning (ERP) systems are leveraged. ICS612 covers these critical systems, and Figure 4 illustrates how ERP and MES systems are logically placed in a manufacturing facility. The MES forms the central nervous systems of the plant floor, pulling operational data from SCADA systems or directly from controllers and then updating controllers with new product run specifications, recipes, and tracking products throughout the manufacturing process. The ERP is the business side of manufacturing and performs order processing, invoicing, inventory, forecasting, and many other business functions necessary to run the plant.

Figure 4: MES and ERP in Manufacturing

Incident Response

In the case of a ransomware event where the incident response plan is activated, one of the first actions is to scope the incident and begin to contain the incident. However, containment often consists of segmenting the IT/OT boundary. In many ICS/OT environments, this action could stop product production as MES systems are tightly integrated between IT and OT. Without the MES, products may not be tracked, labels not printed, and inventory unknown. Thus, understanding these dependencies before an incident is critical, and as I am teaching ICS612, it is one of the many areas I focus on with the class.

Therefore, developing an incident response plan according to a realistic scenario facing your industry, such as ransomware, is a critical first step that asset owners and operators should undertake. As control one of the five critical controls, it's the most important control to have (albeit also one of the easiest controls to implement) because, in this adversary landscape, it's not if but when an incident will happen. Having an IR plan is good, but testing the plan with a Tabletop Exercise (TTX) is equally essential to understand if the document is fit for purpose and if the response teams know their roles, are trained, and are ready to respond from an organizational and technical perspective.

ICS612 students learn the importance of training for an incident by taking backups, practicing system containment, looking for signs of malicious tampering and system abuse, and restoring systems back to a known good operational state. When ransomware strikes, it can be devastating for companies to respond to, especially if they have not trained for the scenario. At this point, it's too late to understand if backup processes are weak or if reliance on always-connected backup systems proved in error when it becomes the primary adversary target before going after the networked attached hosts. These topics and more are brought to life when students try to recover their compromised PLCs and HMIs during the Covfefe Down exercise. During the day five capstone event, students harness all aspects of the course lectures and labs and engage in a hands-on cyber-physical exercise to restore the Covfefe plant for Java production.

Secure Remote Access

Apart from securing the IT/OT boundary is developing the infrastructure and capabilities to establish a micro-segmented DMZ. The DMZ is more of a logical construct that allows services, such as remote access, for owners and third parties to leverage. Secure Remote Access is the fourth critical cybersecurity control and a common pathway for adversaries.

When not teaching for SANS, I am a Technical Consultant Lead for Dragos. A frequent finding I come across in customers' environments is poorly implemented remote access solutions. In ICS612, there is a remote access lecture as well as a lab to aid in knowledge retention. The lab leverages the Windows Remote Access server to provide students with an example of setting up a remote access solution at their company. Many other vendor solutions exist for remote access, and I recommend their consideration. Still, the remote access cybersecurity control can be implemented on a Windows server - of course, with multiple configuration steps and access to a certificate authority.

ICS Network Monitoring and Visibility

Day four of ICS612 covers the all-important topic of host, network device, and network traffic visibility, which is the third critical cybersecurity control. As I teach this section, I often repeat the phrase from Dr. Eric Cole, which I first heard while taking SEC401 many years ago, that "Prevention is ideal, but detection is a must." For ICS/OT security, this phrase couldn't be more accurate. Monitoring host logs and network visibility across the OT environment is critical to understanding if adversarial activity is in your environment and responding if detection occurs.

Lecture materials and labs cover using both open-source tools and paid vendor tools to allow students to see the difference between them but also to allow those with varying Opex and Capex budgets to deploy monitoring solutions back at their places of work. Ransomware implantation detection is much more obtainable by monitoring for North-South communications across the IT/OT boundary and collecting host logs across the DMZ and Layer 3 Purdue model systems. However, monitoring doesn't stop here.

Monitoring network communication across the East-West plane is also critical where workstation-to-controller or server-to-controller communications occur. Additionally, the host logs from these systems can be forwarded using native Windows Event Forwarding to a Windows Event Collector server and sent to an IT/OT Security Information and Event Manager (SIEM) for SOC analysts to monitor. My white paper, Gaining Endpoint Log Visibility in ICS Environments, further delves into the details of Windows Event Forwarding.

Vulnerability Management

History shows even the most defended castles can eventually be breached due to vulnerabilities found within the walls or internal layout of the fortress. ICS/OT environments are no different, and vulnerabilities found in devices at the perimeter and interior need to be understood and dealt with.

This brings us to the fifth critical cybersecurity control of key vulnerability management. For ICS612, we discuss strategies for dealing with vulnerabilities, such as considering the edge first, where high IT/OT interconnectivity occurs, and then looking further down in the Purdue levels to consider Crown Jewels and other critical systems.

As shown in Table 1 below, rolling out patches or other mitigating means should be done using vendor-approved patches, per allowed operational and maintenance windows, and at a staged rollout, such as patching one set of a redundant server pair or half of the operator consoles at a time.

Table 1: Example ICS/OT Patch Strategy

This is due to the likelihood, albeit low, that a patch may disrupt an ICS/OT application that could affect the visibility or control of the plant floor. Many adversaries leverage unpatched and vulnerable systems to deploy ransomware. At control number five, vulnerability management is one of the areas that companies must continuously focus on. However, it's important to consider that this should not be the primary focus area.

Wrapping up

Ransomware will continue to be effective if we let our guard down and not defend our own. A well-defended castle is not easily breached, and we should leverage the SANS Five Critical Cybersecurity Controls to create a defensive posture against such adversary Tactics, Techniques, and Procedures (TTPs).

SANS ICS612 will allow you and your team to consider the entire ICS/OT security stack, from the process to controllers, servers, and workstations to remote access, logging and monitoring, backup and restore, and beyond.

Continuous learning is a responsibility for all of us employed in the vital role of defending critical infrastructure, and ICS612 provides an exceptional opportunity to take your knowledge and skills to the next level.

I look forward to seeing you in class.

To learn more about how to effectively defend your industrial environments from ransomware and leverage the Five ICS Cybersecurity Critical Controls, download the SANS Strategy Guide: ICS is the Business. This guide provides actionable insights and best practices to fortify your defenses and ensure operational resilience.