16 InfoSec Courses, 2 Weeks of Training at SANS Virginia Beach 2017. Save $400 thru June 28.

IDFAQ: Network Intrusion and use of automated responses

By: Dan Hawrylkiw

I overheard a whispered conversation at the San Diego SANS Network Security 2001 Conference that started after David Hoelzer reviewed the signatures of the December 25th, 1994 attack on Tsutomu Shimomura‚s home network. As the presenter started the review, we were asked to remember ‚16 seconds‚. Only after reviewing the signatures and results of the attacks that were used to gain root access to one of Shimomura‚s workstations, did he explain that the event unfolded in only 16 seconds. The first whisper stated, ‚An analyst sitting at a console wouldn‚t be able to stop that attack‚. The reply was, ‚He was out of town; there was no way to prevent it anyways‚. The conversation touched on automated response, but none of the parties involved could agree on the practicality of automating responses to the attack that was, at the time, only theoretical. At that date- practical responses did not exist. And today, the practicality of automating responses still isn‚t clear.


Network Intrusion Detection is the art and science of monitoring networks for activity that may jeopardize the security of the infrastructure under surveillance. By definition and design, this is a detective tool that improves upon the tedious review of logs or recorded traffic by sniffing network traffic and filtering for specific events, typically in real-time. When suspicious traffic is detected, the minimum actions of the Network Intrusion Detection Systems (NIDS) are to log the event and/or contact the appropriate personnel.

While Intrusion detection systems reduce the time it takes to identify suspicious activity, further actions have been dependent on human intervention to begin a response for three reasons. First, Network Intrusion Detection Systems are imperfect and can alert on non-malicious traffic, resulting in false positives. Second, not all legitimate alerts warrant a response. Third, most alerts that warrant a response, require human judgment to determine the most appropriate action. Therefore an analyst is required to further validate alerts and decide if, and how, to take any actions. Unfortunately, the human interaction is the most time consuming element in an attack response cycle.

Modern NIDS can initiate responses in addition to simple notifications. These responses usually fall under direct intervention or scripted reconfiguration of surrounding equipment. An automated response does not necessarily need to address the traffic directly, but could assist the engineers in handling incidents with greater efficiency.

Types of responses

Session Sniping

Direct intervention to disrupt communications between an attacker and victim is often called session sniping or knockdown. This action is performed by injecting packets to break down a connection that triggered the response. The most effective knockdown for a TCP connection is to forge packets to reset the connection. To do this, the NIDS must forge packets to send to one or both systems with the TCP Reset bit set. The source IP, Ports, and sequence numbers must be in sequence with the traffic that triggered the event for the reset to be effective. This response is integrated with or in development for IDS systems that are currently on the market today.

Injecting resets cannot guarantee that the session knockdown will be successful. One reason is that the TCP/IP stacks of the victim and attacker systems may handle the forged resets in different ways. Some TCP/IP stacks will only accept the first packet with the correct sequence number, while ignoring any replacements. Others may overwrite the previously received packets with the latest packet matching the sequence. Because the forged packets are injected into the network in sequence (or in the next sequence) and may be competing with additional traffic from the same session, one cannot assume that the reset packet will be heard by the OS.

ICMP Messaging

Protocols such as UDP cannot be knocked down by sending resets because of their connectionless nature. Since ICMP and UDP do not support transport layer flags to close connections, malicious traffic cannot be stopped by injecting packets without involving higher layer controls. To accommodate this, without forging packets with higher layer payloads, ICMP error messaging can be used. This response sends an ICMP error message to the attacker identifying that the victim, the victim‚s network, or the destination port is not available. The intention is to have the attacking host believe that the victim cannot be reached, regardless of any other traffic received.

ICMP messaging seeks to notify the attacker‚s TCP/IP stack that the victim in unavailable in some way. Unfortunately, the likelihood that this message will be received, understood, and followed by the attacking machine is low. Many network attack tools do not use the operating system‚s TCP/IP stack, and definitely aren‚t written to be RFC friendly. This extends to the simple hackers attacking from home with consumer equipment. Most home and small business firewalls, whether host or appliance based, don‚t bother with ICMP error messaging and drop the message on ingress. This increases the likelihood that the ICMP message will be ineffective in stopping an attack.


Shunning is the denial of access to a host suspected of originating an attack. Access can be denied at a target host or at a network chokepoint; and can block hosts, networks, or connectivity to specific services. A common response is to block access to an attacker‚s IP at an ingress point to reduce the possibility of expanding the attack to other targets within the protected environment.

Firewalls are an ideal location to deny access to a site. The use of scripting or plug-ins such as Checkpoint‚s Open Platform for Security (OPSEC) allow for simplified NIDS/firewall integration by allowing the NIDS to directly edit the firewall rules. This allows the perimeter access controls to be updated in near-real time by the triggering of events on the Network Intrusion Detection System.

While this may sound ideal, shunning is often avoided because it provides additional control to the attacker. An attacker who identifies a site that is shunning suspicious sources may decide to send forged packets, forcing your firewall to deny services to legitimate users. This was defined by Marty Roesch, author of the Snort NIDS, as ‚putting the attacker in control of the firewall‚.

Non-Blocking Responses

The most frequently discussed active responses are blocking countermeasures that intervene directly with either the traffic or the path taken by the traffic. However, non-blocking responses can also be used to protect a computing environment. Most of these responses are innocuous and may seem transparent to the suspect user/system.

Post attack cleanup is possible with the assistance of a NIDS. This response would involve a scripted action to execute upon detection of an attack. While it is unlikely that this could be used to protect from an attacker manually breaching a system and using a variety of tools in infinite number of ways, these scripted responses could address well-known attacks with well known fixes. For example, one of Nimda‚s attack vectors is to create .eml files on shares, expecting that an unsuspecting victim will attempt to open them with a Windows system. The NIDS, upon detection of this activity, could launch a script to delete the .eml file from the share, scan the share for more suspicious files, and possibly initiate a virus scan on the infected host.

Another non-blocking response that is commonly associated with shunning is redirection. This can be used to protect networks by redirecting suspicious hosts through additional security controls or by changing destinations altogether. In cases where it is not practical to shun hosts, it may be reasonable to redirect attackers through alternate firewalls with more restrictive ACLs. One interesting derivative of redirection is transferring the attacker to a honeypot. This can allow the security administrator to monitor attacks in detail- allowing the attacker to further attack the system without risking production servers.

Extended Notification

One of the basic expectations of a Network Intrusion Detection System is that it must be able to quickly filter and log events of interest. In the interest of reducing response time, most systems have a mechanism to ‚push‚ notifications to NIDS console administrators or analysts. While the basic capability might not be considered an active response, extending the notification to identify and contact external entities is an option that could be scripted. While most contacts should be reviewed first, well-known attacks could be automatically escalated to the attacker‚s ISP.

Risky Business

The ability to automatically attack back exists, but is not listed as a legitimate response because far too many pitfalls exist for this to be a real-world option outside of a laboratory. Because a NIDS, or human in some cases, cannot quickly discriminate a true attacker from a backdoored host or spoofed traffic source, the possibility of attacking an innocent system exists. Along the potential legal pitfalls of attacking the suspected host, legal precedent exists stating that carriers can hold the ‚original victim‚ responsible for their counteractions and potential effect on the carrier‚s systems. Aside from the issues of counter attack; this is also an ideal way to tip of an attacker, taint evidence, and ruin an investigation.

Food for Thought

While active responses sound great on paper and make great sales pitches, they are not bulletproof solutions. The risk of denying legitimate access to services increases with the use of active responses, especially persistent countermeasures. Automated responses may only work effectively in specific scenarios or on limited traffic types.

Worm and Virus Attacks

The recent success of worm attacks has clearly identified that, in some cases, requiring any human interaction is not fast enough to successfully contain malicious traffic. The infection rates of worms such as CodeRed and Nimda allow the worms to propagate throughout a network in seconds or minutes, shifting the response from containment to cleanup.

Because of the mechanism of infections, session sniping is usually ineffective in stopping these attacks because of the timing and small number of packets involved. If the buffer overflow or command is issued in a single packet, the NIDS will not be able to intervene before the damage is done. For example Appendix A identifies a packet trace from a simulated worm attack that failed to intervene. Additionally the practicality of resetting sessions at a rate of hundreds or thousands of packets per second is likely to overwhelm the NIDS.

Fast, automated responses can help to contain the outbreak. The most effective approach is to ‚reach ahead‚ of the attack, by creating network ACLS that block the suspicious traffic or to drop the infected host off the network (perhaps applicable to clients only). This is far less intrusive than shutting down the network or knocking critical servers off the network. For this, signatures can be used to identify well-known attacks, and new attacks could be identified by excessive scanning or expected traffic patterns.

A key requirement for using an automated response to contain an outbreak is assessing the risks involved. An automated response is practical only when the risk of denying legitimate services is far outweighed by the risk of infection, damage, and loss of service.

Limited Shunning

In open networks, such as Universities, where the networks are not isolated from the Internet by a firewall; the networks are an open playground for hackers. Not only are the typical services (Web, SMTP, FTP, etc) available, but all ports and protocols are exposed to attack or misuse. If these networks utilize shunning, similar to a firewalled site, they still give too much control of their perimeter to the attackers.

If shunning is used, it could be restricted to non-standard traffic. This is much less likely to allow the attacker to initiate denial of service attacks by triggering block/shun responses, and still blocks a majority of the exposure points.


Automated responses are a powerful extension to the art of Network Intrusion Detection. They allow the systems to respond to incidents much faster than any human could, by following prescribed steps when triggered.

The risks associated with automated response can be extremely high if not configured correctly. The existence of vague rules and false positives dictates that automated responses are not appropriate for most traffic. Specific traffic types can be addressed, but the risks of denying access to legitimate services must be reduced below the risk of loss from an attack. For this to be true, the Intrusion analysts must have a firm grasp upon the signatures used to identify events. As with any major change, proper review and testing should be done to reduce the possibility of interruption, failure, or escalation of the event.

Appendix A. - Failed TCP Reset Attempt

The following is an example of a TCP reset response issued by a Snort 1.8.3 NIDS after triggering on a ‚WEB-IIS ISAPI .ida? request‚. The alert, and subsequent response, is triggered upon detecting the string ‚.ida?‚ in the HTTP GET request from the attacker.

The output was generated from TCPDUMP 3.6.2. The attacker‚s IP is, and the victim is at Blue packets indicate traffic between the attacker and the victim. Red Packets were generated by Snort in response to the simulated attack, and only attempt to reset the connection at the victim. The 3-way handshake has been removed for brevity. The simulated attack is executed in packet #4.

05:23:15.113298 0:10:67:0:85:7a 0:90:27:8c:e9:f3 ip 536: > P 4211792473:4211792955(482) ack 3225804639 win 17520 (DF) (ttl 51, id 11759, len 522)
0x0000      4500 020a 2def 4000 3306 cf19 8fb7 980a   E...-.@.3.......
0x0010      d0ba 5069 e54e 0050 fb0a da59 c045 df5f   ..Pi.N.P...Y.E._
0x0020      5018 4470 9648 0000 4745 5420 2f64 6561   P.Dp.H..GET./dea
0x0030      6675 6c74 2e69 6461 3f4e 4e4e 4e4e 4e4e   fult.ida?NNNNNNN
0x0040      4e20 4854 5450 2f31 2e30 0d0a 4163 6365   N.HTTP/1.0..Acce
0x0050      7074                        pt

05:23:15.123298 0:90:27:8c:e9:f3 0:10:67:0:85:7a ip 60: > . [tcp sum ok] 3225804639:3225804639(0) ack 4211792955 win 31638 (DF) (ttl 63, id 59367, len 40)

05:23:15.123298 0:90:27:8c:e9:f3 0:10:67:0:85:7a ip 572: > P 3225804639:3225805157(518) ack 4211792955 win 32120 (DF) (ttl 63, id 59368, len 558)
0x0000      4500 022e e7e8 4000 3f06 08fc d0ba 5069   E.....@.?.....Pi
0x0010      8fb7 980a 0050 e54e c045 df5f fb0a dc3b   .....P.N.E._...;
0x0020      5018 7d78 2a7f 0000 4854 5450 2f31 2e31   P.}x*...HTTP/1.1
0x0030      2034 3034 204e 6f74 2046 6f75 6e64 0d0a   .404.Not.Found..
0x0040      4461 7465 3a20 5375 6e2c 2030 3220 4465   Date:.Sun,.02.De
0x0050      6320                        c.

05:23:15.123298 0:90:27:8c:e9:f3 0:10:67:0:85:7a ip 60: > F [tcp sum ok] 3225805157:3225805157(0) ack 4211792955 win 32120 (DF) (ttl 63, id 59369, len 40)

05:23:15.123298 0:90:27:8b:17:17 0:90:27:8c:e9:f3 ip 60: > R [tcp sum ok] 4211792473:4211792473(0) ack 3225805121 win 0 (ttl 255, id 22081, len 40)

05:23:15.123298 0:90:27:8b:17:17 0:90:27:8c:e9:f3 ip 60: > R [tcp sum ok] 4211792473:4211792473(0) ack 3225805121 win 0 (ttl 255, id 22081, len 40)

05:23:15.243298 0:10:67:0:85:7a 0:90:27:8c:e9:f3 ip 60: > . [tcp sum ok] 4211792955:4211792955(0) ack 3225805158 win 17520 (DF) (ttl 51, id 11796, len 40)

05:23:15.253298 0:10:67:0:85:7a 0:90:27:8c:e9:f3 ip 60: > F [tcp sum ok] 4211792955:4211792955(0) ack 3225805158 win 17520 (DF) (ttl 51, id 11799, len 40)

05:23:15.253298 0:90:27:8c:e9:f3 0:10:67:0:85:7a ip 60: > . [tcp sum ok] 3225805158:3225805158(0) ack 4211792956 win 32120 (DF) (ttl 63, id 59370, len 40)

The victim server acknowledges the attacker‚s request in packet 5, and the server replies with a 404 (not found) in packet 6. The server even had time to issue a FIN packet before the NIDS injected a Reset (approx 20 mS after the ‚attack). Packets 10-12 indicate a normal session FIN. The forged packets are further identified as unique by the source MAC address 0:90:27:8b:17:17, which is the hardware address for the Snort NIDS interface.

It is worth noting that the forged packets used the same sequence number (4211792473) as the attack. This assumes that the Operating System‚s TCP/IP stack will overwrite or replace any traffic received with the same starting sequence for the reset to be effective.


1. Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection (1998) ‚ Ptacek and Newsham

2. Intrusion Signatures and Analysis (2001) - Northcutt, Cooper, Fearnow, Frederick. ISBN: 0-7357-1063-5

3. Know Your Enemy, Reveling the Security Tools, Tactics, and Motives of the Blackhat Community (2001) ‚ The Honeynet Project ISBN: 0-201-74613-1

4. Network Intrusion Detection, An Analysts Handbook 2nd Edition (2001) - Northcutt and Novak. ISBN: 0-7357-1008-2

5. Snort Users Manual: Snort Release 1.8.3 (2001) ‚ Martin Roesch

6. TCP/IP Illustrated, Volume 1 (1994) - W. Richard Stevens ISBN: 0-201-63346-9