Security Training
Security Certification
Cyber Security Graduate School
Internet Storm Center
Security Awareness Training
Computer Forensics
Penetration Testing
IT Audit
Software SecurityIntrusion Detection FAQ: What is the Role of Security Event Correlation in Intrusion Detection?
|
Introduction Millions of dollars have been invested in security products such as firewalls, intrusion detection, and strong authentication over the past several years. However, system penetration attempts continue to occur and go unnoticed until it is too late. As a consequence financial losses continue to skyrocket for organizations. According to the 2002 CSI/FBI Computer Crime and Security Survey, average losses per respondent topped $2,000,000 for the year! It is not that security countermeasures are ineffective against intrusive activity. Indeed, they can be very effective within an organization where security policies and procedures require analysis of security events and appropriate incident response. However, as pointed out by Steven Northcutt of SANS, deploying and analyzing a single device in an effort to maintain situational awareness with respect to the state of security within an organization is the "computerized version of tunnel vision" . Security events must be analyzed from as many sources as possible in order to assess threat and formulate appropriate response. Extraordinary levels of security awareness can be attained in an organization's network by simply listening to what its devices are telling you. This paper will demonstrate to intrusion analysts why correlative analysis must occur in order to understand the complete scope of a security incident. Correlation Simplified When law enforcement agents investigate a murder, they do more than examine the body for clues. The investigative process calls for searching the surrounding crime scene, interviewing individuals who know the victim, and soliciting requests to the public for anyone who might have information related to the crime. A similar process should apply to intrusion analysis. If your web server is attacked, analyze more than the web server logs. Search the firewalls and intrusion detection systems protecting the web server for other activity from the source address. Share your log information about the activity with other analyst via websites such as incidents.org. Reviewing all of the information collectively provides a more complete picture of the incident and assists in answering the who, what, when, where, and why's of an attack. Correlation Demonstrated Understanding the concepts of correlation can be dramatically simplified if the responses of various network devices are examined in the face of a probe or attack. The following scenario demonstrates how independently obscure security events can be correlated from multiple logs, and in doing so provide the higher level of vision necessary for accurate and expeditious intrusion analysis. We will conduct intrusion analysis of the log data independently and the collectively to show how a more complete picture can be attained trough correlative analysis. The network below depicts a typical network layout where common countermeasures such as firewalls and intrusion detection systems are deployed.  In this network, we have established multiple log generators of interest (moving from the perimeter inward):
We will analyze the log response of all of the network devices while Attacker1 is launching a series of probes searching for exploitable CGI scripts. This activity is being conducted by an attacker at 152.63.146.6 against an Apache web server (www) running on a typical Linux distribution. For this exercise, we will confine the probes to three well known exploits:
First, let's review the log activity related to the probe activity for each device in the path of the probes. We will first analyze the information independently and later we will correlate all of the log data for a more complete picture of the incident. Router Logs (Cisco):
For the source address 152.63.146.6, we observe that on May 31 at 09:27 a series of connect attempts occurred directed towards the xxx.yyy.zzz.0/24 network. This log data has been pruned but other entries show the activity directed towards the entire class C. By the destination TCP port 80 connection attempts, it appears as though 152.63.146.6 is conducting a broad scan searching for web servers. It is interesting to note that there was no denied log entry for an access attempt to xxx.yyy.zzz.4 because this is the web server in our network. Our router access control lists have been configured to allow inbound TCP port 80 traffic with ephemeral source ports to xxx.yyy.zzz.4 because this is our company web server. By only looking at the router logs, the information presented to us suggests 152.63.146.6 swept the entire class C looking for web servers. Independently reviewed, we have no other insight into the intentions of the activity. In summary for the router logs: Who: 152.63.146.6 What: Broad scanning of xxx.yyy.zzz.0/24 network for web servers. Likely found xxx.yyy.zzz.4. When: May 31 at 09:27-09:28 Where: Company DMZ network Why: Likely reconnaissance. Firewall Logs (Gauntlet):
For the source address 152.63.146.6, we observe that on June 1st, a series of http connects were allowed to the corporate web server. We see that the URL path shows attempted access on three separate cgi scripts: phf,formmail, and survey.cgi. Reviewed independently, we have no way of knowing if the access attempt was successful. All we know is an attempt was allowed. And unless we are versed in known cgi vulnerabilities, we may simply overlook the activity as legitimate. In summary for the firewall logs: Who: 152.63.146.6 What: Three http connects to xxx.yyy.zzz.4 with access attempts of cgi scripts: phf, formmail, and survey.cgi. Unknown if the scripts were accessed. There were no other http connections from this host so it appears as though this is malicious activity not associated with any other normal web traffic. When: June 1 at 06:08:50 Where: Company DMZ network Why: Research shows that phf, formmail, and survey.cgi are all exploitable scripts. These access attempts in isolation suggest malicious activity because if accessed, these scripts could allow remote command execution. IDS Logs (Snort):
For the source address 152.63.146.6, we observe that on June 1st, a series of cgi access alerts occurred. The alerts point to vulnerabilities associated with these scripts that can be used for remote command execution. These are the Snort rules that triggered these alerts:
These alerts only trigger when certain strings occur in a URL. These types of alerts are known for false positive alerts. Reviewed independently from other devices, we have no way of knowing if these access attempts were associated with other legitimate access and therefore false positives. However, we can infer that this is malicious because it would be highly irregular for these three scripts to be accessed by back-to-back-to-back connection attempts as indicated by the ephemeral source ports. Even so, we do not know if the attempts were successful or unsuccessful. All we know is that the attempts occurred. In summary for the IDS logs: Who: 152.63.146.6 What: Three http connects to xxx.yyy.zzz.4 with access attempts of cgi scripts: phf, formmail, and survey.cgi. Unknown if the scripts were accessed. There is no other log information available to evaluate if this is a false positive other than the fact these three scripts are unlikely to be accessed within this short of a time frame with incrementing ephemeral source ports. When: June 1 at 06:08:50 Where: Company DMZ network Why: If these vulnerable scripts are in operation on the web server, they would allow remote command execution by an attacker. Web Server Logs (Apache): access_log
error_log
For source address 152.63.146.6, we find log entries in both the access_log and error_log. The access log shows that on June 1st at 06:08, attempts to access cgi scripts phf, formmail, and survey.cgi in the cgi-bin subdirectory occurred. The error_log shows that this activity generated errors because these scripts were not in operation on this server. There were no other http connections from this host so it appears as though this is malicious activity not associated with any other normal web traffic. In summary for the web server logs: Who: 152.63.146.6. Likely a Unix/Linux host using Lynx v2.8.5dev.2 as the tool to conduct the activity. What: Three http connects to xxx.yyy.zzz.4 with access attempts of cgi scripts: phf, formmail, and survey.cgi. The scripts were not accessed because they could not be found on the server. There were no other http connections from this host so it appears as though this is malicious activity not associated with any other normal web traffic. No other access log entries for 152.63.146.6 suggests this activity is malicious. When: June 1 at 06:08:50 Where: Company DMZ network Correlative Analysis We demonstrated above that attempting to understand the full scope of a security incident is encumbered if logs and alerts from only a single device are analyzed. Each device has its own limits as to what it can tell us in the analysis process. However, collectively analyzed, the picture becomes much clearer. Let's take a look at what we can determine. There are two separate episodes of activity that comprise the total picture of the security incident perpetrated by 152.63.146.6.
 The diagram shows that removing the analysis of even just one of the device's log data, our understanding of the incident can drop dramatically. For example, if we remove the analysis of the web server error_log, we would not have known that the script access attempt failed. If we had not analyzed the router, we would not have known the probing host scanned the entire class C of addresses for web servers. If we had not analyzed the www access_log, we would not have known that the probing host was likely using Lynx as the web browser to check for the scripts. If we had not analyzed the network IDS logs, we may not have known that the activity was related to well known exploit attempts. Conclusion Analyzing a single device to in an attempt to conduct intrusion analysis is the "computerized version of tunnel vision" . Security events must be analyzed from as many sources as possible in order to assess threat and formulate appropriate response. Extraordinary levels of security awareness can be attained in an organization's network by simply listening to what its devices are telling you. This concept was demonstrated by examining how security events reviewed independently only paint part of the picture. However, when the correlation of event data across platforms occurs, a more clear understanding of the scope of security incidents is attained. References Curtin, Matt and Marcus Ranum. "Firewalls FAQ" URL: http://www.faqs.org/faqs/firewalls-faq/ (May 31, 2002). "Interpret Syslog and Console Messages Generated by C ontext-Based Access Control." The Cisco IOS Firewall Feature Set and Context-Based Access Control. URL: http://www.cisco.com/univercd/cc/td/doc/product/software/ios113ed/113t/113t_3/firewall.htm (May 31, 2002). "Log Files." Apache HTTP Server Version 2.0. URL: http://httpd.apache.org/docs-2.0/logs.html (June 1, 2002). Northcutt, Steven. "Coordinated Attacks." IDS Signatures and Analysis-GCIA Courseware. 2001 Power, Richard. 2002 CSI/FBI Computer Crime and Security Survey. Vol. III, No. 1, Spring 2002. Snort Users Manual - Snort Release: 1.9.x. URL: http://www.snort.org/docs/writing_rules/index.html (June 1, 2002). Steven Drew |
Check Them Out!
I have attended many conferences/training sessions, and SANS by far has been the best. The instructors are the top in the industry, examples are from real life experiences - terrific!
-Chris Bush, Novartis Pharmaceuticals
