Active Response is a mechanism in intrusion detection systems (IDS) that provides the IDS with capability to respond to an attack when it has been detected. There are two methods that the IDS can take to circumvent an attack. The first method of circumventing attacks would be Session disruption, and the second is Filter rule manipulation. The specific feature varies with each IDS product and each countermeasure method possesses its own strengths and weaknesses.
Method 1- Session disruption
Session disruption is the most popular method of circumvention because of the ease of its implementation. Depending on the type of session established, UDP or TCP, an IDS that is configured for session disruption can reset or knock down the established connection. This does not prevent the attacker from launching additional attacks, but it does prevent the attacker from causing any further damage in conjunction with the "broke" session. When using the session disruption method, if an attacker launches subsequent attacks, the IDS must continually attempt to close every initiated attack session.
With sessions disruption the IDS uses different methods for breaking the connection depending on the type of traffic it sees. If an attacker uses TCP sessions, they are reset by RST packet that is sent to reset one or both hosts in a session from the IDS. In the case of UDP, a session can be broken by sending various ICMP packets to the host from the IDS box.
Why might the IDS send RSTs to the attacker and victim host?
An IDS might send a TCP RST packet to an attacker and victim after detecting malicious traffic like an established Sub seven connection.
There are a few IDS systems that provide the session disruption, but for discussion I will focus on Snort, which is a lightweight network intrusion system that runs on different platforms. When Snort is configured with the Flexresp feature it provides session disruption. Flexresp is a feature that allows Snort to automatically respond to an attack if the corresponding option is specified in the snort rule. In order to enable active response on Unix, Snort must be compiled with Flexresp enable as shown below.
When installing on a Win32 system, Flexresp is enabled by selecting the Snort +FlexResp option as shown in Fig 1.1 below.
Below in Fig 1.2 is an example of a Snort rule configured to respond to an attack
Rules define what traffic snort considers as hostile and consists of two parts, Rule Header and Rule Options. The Rule Header contains an action field, protocol field, source IP and port fields, direction of traffic field and destination IP and port fields, which all basically define who is involved. The Rule options define what packet attributes to search to consider if traffic is hostile.
When we examine the Rule Header of the first rule in Fig 1.2 we see that Snort will alert us to any TCP session connecting to the web server at port 80. Let's look at the second part of the rule, the Rule Options. Snort inspects packets that meet the Rule header requirements for the TCP ACK flag and any other TCP flags that are set, and searches the payload for the character string scripts/root.exe. The resp:rst_snd value sends a forge packet with the TCP reset flag set to the sender.
The second rule is for any Webtrends UDP Scans with the character string of "0A 68 65 6C 70 0A 71 75 69 74 0A" in the payload. If this rule meet both the header rule and option rule the flexresp values icmp_port,icmp_host tells snort to send an icmp port unreachable and host unreachable packet to knock down connection.
Why does Snort send ICMP packets to UDP stimulus?
ICMP packets are sent to a host initiating a UDP connection to inform the sender that a requested port/host is unavailable. The reason ICMP packets are sent to a UDP stimulus is UDP does not have the capability to report errors, so ICMP is used to assist. Snort use this normal process to send a spoofed ICMP packet to the host initiating the connection in an attempt to fool the host in to thinking that the host is unavailable.
Session disruptions in action
alert tcp 192.168.1.1 any > $HOME_NET 135 (msg:"Block host"; flags:S+; resp:rst_snd;)
This rule was created to rest any TCP session initiated by host 192.168.1.1 with the SYN TCP flag and any other TCP flags set.
The traffic below was generated in my lab between two machines. The targeted pc is configured with Snort 1.8.3 for Win32 systems and runs on windows 2000 professional. The attacking host is a Red Hat Linux 7.0 machine. Nmap was used to port scan the target machine by typing nmap -p 135 -sF 192.168.1.2, which triggered alert.
08:17:23.477034 Attacker.4634 > Target.135: S 3719449388:3719449388(0) win 5840
08:17:23.477203 Target.135 > Attacker.4634: R 0:0(0) ack 3719449389 win 0
08:17:23.477275 Attacker.4635 > Target.135: S 3715810638:3715810638(0) win 5840
08:17:23.477346 Target.135 > Attacker.4635: R 0:0(0) ack 3715810639 win 0
There are a few techniques which can be used that allow an attacker to bypass session disruption enable IDS. An attacker with basic knowledge of TCP/IP can defeat this mechanism as stated in a paper by Jason Larsen and Jed Haile on Understanding IDS Active Response Mechanisms. Here they wrote about techniques that could defeat session disruption. One of the methods they talked about was trying to have the host disregard the tcp reset packet sent from the IDS system. The session disruption bypass techniques took advantage of the time it took for the IDS to examine network traffic, detect an exploit and respond to an attack. Also the tcp stack and the way it receives data were used to circumvent session disruption.
An attacker could also attack the IDS with a Denial of Service in an attempt to crash the machine or starve it of it's resources and render the use of session disruption. Any Evasion techniques where an attacker tries to prevent the IDS from detecting the rule would also work. Session disruption is only useful when the IDS can identify the traffic.
Method 2- Filter rule manipulation
The second countermeasure is filter rule manipulation. This mechanism works by modifying the access control list (acl) on a firewall or router. Filter rule manipulation block the IP of the attacker preventing any further attacks. This option should be used with extreme care, because an attacker can use it to Dos the network. If an attacker used the IP address of a partner they could spoof the address. When the IDS sees the attack and goes to respond, it would block your partner access.
There are a few IDS products that provide filter rule manipulation. Real Secure has the ability to modify Checkpoint firewall. Cisco Intrusion Detection System (IDS), formerly known as Cisco NetRanger, is a hardware based IDS that can respond to an attack by adding an access control list to a router.
Snort can provide filter rule manipulation when used with IDScenter, a tool used to manage snort, when run on a Win32 systems and BlackICE Defender. Attackers are blocked by IDScenter after an alert is triggered which modifies the file firewall.ini access lists that is used by BlackICE Defender, a personal desktop firewall that only protects the machine it is installed on. This can be accomplished by checking on the IDScenter Auto block box as shown in Fig 1.3 below.
Then you provide the path to the firewall.ini file that is used by BlackICE, which is C:\Program files\Network ICE\BlackICE by default.
This method can be evaded by tricking the user behind the firewall in to installing a backdoor via email. Once the backdoor is installed the attacker can remotely admin PC and can launch attacks from within. Jason Larsen and Jed Haile wrote paper on Understanding IDS Active Response Mechanisms mentions launching an attack with a spoof address of an popular website like CNN.com, AOL.com, and Ebay.com. This would block traffic from these sites to enter your site. Users would call the helpdesk about not being able to access site and demand a resolution. This would result in the disabling of the rule manipulation feature allowing the attacker to attack without blocking.
Active Response mechanisms is an effective tool when used within its limitation. When used in conjunction with other network security devices it enhances network security. Session disruption should not be configured to respond to every alert just serious attack like Denial of service. Rule manipulation should be used with care because of the effect it could cause if turn on a network. Active Response is by no means meant to be fool proof.
Larsen, Jason, & Haile, Jed. Understanding IDS Active Response Mechanisms
Brenton, Chris. Mastering Network Security. Pages 263 - 265
ISS RealSecure 6.5 FAQ
Cisco Secure Intrusion Detection Family Overview
Roesch, Martin. Intrusion Detection: Snort Style. Pages 171 - 179
Ptacek, Thomas H, Newsham, Timothy N. Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection