March 4, 2001
In order to satisfy Security Professionals’ needs to monitor their networks, an abundance of Enterprise Management Systems (EMS) have surfaced that tout themselves as "All-in-one" Network Monitoring systems. These frequently have a component that performs some level of Network Intrusion Detection and that displays the alerts from the intrusion detection (ID) sensors on the same console as other Monitored Services, Networks or Systems. While many of these products are indeed quality applications, the resulting integration with general monitoring services may be undesirable, giving Monitoring Operations Staff the ability and responsibility of analyzing Intrusion Incidents- something for which entry-level engineers generally are not qualified. In addition, these multifarious application suites generally require higher-end equipment, which may have the effect of a company choosing far fewer sensors than it might if sensors could be built from obsolete stock or far less expensive new equipment. An EMS can be expensive even without the necessary costly equipment, which must also be added to the price of the Operating System needed to run the applications on the sensors and console. To alleviate these issues and give Security Teams the ability to maintain complete control of their NIDS, a variety of solid open-sourced software can be loaded onto low-power, relatively inexpensive equipment while providing the same features as a commercial EMS NIDS component. This document can be looked at as a guideline for such an implementation.
2. What IS Open Source?
Open source software is software like any other; the difference is in the licensing philosophies:
From the preamble of the GNU Public License:
The licenses for most software are designed to take away your freedom to share and change it.Concurrently, the Open Source Initiative (OSI) website claims:
The basic idea behind open source is very simple. When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. People improve it, people adapt it, people fix bugs. And this can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing.Open source licenses are meant to make the distribution of source code for applications free for everyone. They protect the author by stipulating that redistributions of the software must include the full source code, as well as credit to the original author. They basically give everyone the right to share and change the original source code, provided they do not attempt to infringe on the rights of others to do the same. To help maintain standards, the Open Source Initiative offers an Open Source Definition and also serves as a certifying authority for open source licenses. It can be viewed at http://www.opensource.org/docs/definition.html. OSI also maintains a list of accepted open source licenses; for our purposes, these will be:
The PHP License – http://www.php.net/license/2_02.txt
The Apache Software License – http://www.opensource.org/license/apachepl.html
The GNU Public License (GPL) – http://www.opensource.org/license/gpl-license.html
Support for software used in this implementation can be supported by a variety of 3rd party consultants as well as the company maintaining the software package (i.e. MySQL support can be purchased from http://www.mysql.com ). This support differs from most companies in that you rarely talk to a "support technician". For example, MySQL maintains a list of developers currently logged in to their system. When you have a problem, you receive this list of developers and their contact information after logging in to the MySQL website. Depending on the level of support purchased (from e-mail only to full 24/7 phone support), you then contact the developer and he/she handles the support request directly. Most open source software is readily available in both stable and development releases; sites that publish open source software include Freshmeat, Sourceforge, and the GNU (GNU’s Not Unix!) organization.
3. What will I need?
Here’s a list of required software packages, with versions and download sites, used for the system described in this document:
4. What about hardware?
This implementation is designed to allow for a multitude of hardware type to be used. In the practical sense, here are the guidelines for each of the 3 components necessary for an Enterprise Deployment.
ID Sensor—The worker bees of the system. They sit in strategic locations and simply listen to all the traffic passing by them and optionally take action based on pre-defined or dynamic rules. For the ID sensors, ease of deployment and replication should be a top priority. Ideally, unused x86 architecture capable machines with speeds preferably greater than 233 Mhz, 96MB of RAM and 2GIG or more of hard drive space can be found in a closet and given a new lease on life as ID sensors. If such hardware is not available, new equipment need not be more than 500Mhz, with the same minimums of RAM and hard drive space as above. In essence, other architectures such as those found in Macintosh or SPARC computers could be utilized as well, provided the engineers had proficiency installing and administrating Linux on that particular type. This implementation used 5 identical Pentium 266 laptop computers with 2.1 GIG hard drives and 96MB of ram.
ID Console---The back end. This is where the data gets gathered, processed, and presented to the end user. This will need to run the following services:
Remote Console---This will be the workstation that the Security Analyst or monitoring team will use to interface with the ID Console. Any workstation capable of running a GUI Web Browser should suffice, although the A.C.I.D. web interface was only tested on Netscape 4, Netscape 6 and Internet Explorer 5. For administering the ID Console, this workstation should have software capable of communicating via the SSH Protocol Suite.
5. How do I do it?
Setup begins with installing Linux on each of the ID Sensors and the ID console. This can be done individually on each machine with a Linux distribution CD or network installation. Optionally, the installation of the ID Sensors can be automated either with an installation script or by mirroring the hard drives of the Sensors from one installation. In the case of an automated installation using Redhat Linux, a server installation should be chosen from the installer options, with all unnecessary applications being left uninstalled, such as Web Server (this will be installed later), DNS server, anonymous FTP server, News Server, etc. See the SANS Securing Linux Step-By-Step guide (available from http://www.sansstore.org ) and the documentation from the Redhat distribution (http://www.redhat.com/apps/support/documentation.html ) for a more comprehensive description. Once installed, the Linux Kernel should be rebuilt using the most current stable 2.2.X version. This can be found at http://www.kernel.org and at this time is version 2.2.18. If possible, it is recommended to build a monolithic kernel as opposed to as modular (see http://linuxdocs.org/Kernel-HOWTO.html). Support for ipchains, the native firewall for Linus 2.2.x systems, should be enabled (see http://linuxdocs.org/HOWTOs/Firewall-HOWTO.html). After rebuilding the kernel, it’s time to finish the lockdown by running Bastille-Linux. What is Bastille-Linux?
Bastille is set of open source scripts designed to harden a virgin Red Hat … installation.This excerpt comes from "Hardening Red Hat Linux with Bastille" by Sean Boran, available at http://www.securityportal.com/cover/coverstory20000501.html, a useful place to begin for those unfamiliar with Bastille-Linux. Please reference http://bastille-linux.sourceforge.net/ for the most recent announcements, changes, documentation and developments. After completing the Bastille lockdown process, FreeS/WAN should be installed and configured on the ID sensors and console (http://www.freeswan.org/freeswan_trees/freeswan-1.8/doc/install.html#install), to enable a secure communications tunnel. Before beginning this step, it is important to note the IP addresses the ID sensors and console will use on the network. After configuring FreeS/WAN and testing network connectivity between the sensors and console, the rest of the applications can be installed. For the ID Console:
Install MySQL client onlyComplete the sensor by installing snort as in part b. of section IV. above. After installation, create the snort data directory structure:
mkdir /storage/snortAnd copy the snort configuration files to the configuration directory:
cp /usr/local/src/snort-1.7/*-lib /usr/local/src/snort/snort.conf /storage/snort/confConfigure Snort for your site:
Set $HOME_NET, $EXTERNAL_NET, $DNS_SERVERS, and enable/disable any plug-ins you decide upon. Be sure to read the documentation accompanying each plug-in you choose for possible side effects. For the example, these are:Test it.preprocessor defrag
snort -b –l /storage/snort/logs –L snort.log –c /storage/snort/snort.confAssuming the test succeeds, it is time to use your strengths to customize the system. For the example, each sensor has these lines appended to snort.conf:
pass tcp $HOME_NET 22 <> $HOME_NET anyThis will cause snort to log every bit of every packet that it sees, except the ssh traffic to $HOME_NET. While desirable, if hard drive space is scarce this option may not be viable for your sensors. In the example, a 2GIG drive would fill in approximately 4 weeks. Rotating the snort.log file daily and archiving it to the console biweekly (using a simple shell script on a cronjob), kept the sensor’s hard drive from filling up.
Edit /etc/syslog.conf, adding these two lines to enable remote logging of alerts to console:
local2.* @console (set console to the ip or hostname of your ID Console)
6. What Now?
Watch, research, modify rulesets, watch, research, modify rulesets, repeat. The A.C.I.D. should begin to receive events immediately. As you identify frequently occurring false positives, the rule can be modified or removed in the /storage/snort/conf file that contains it.
grep –hwill let you know which file contains the offending rule.
After a day or two of watching traffic and eliminating the false positives identified, you can begin a more granular configuration of SWATCH to notify the security incident handler of only potentially threatening events, set time based alerting (so only the handler on duty receives alerts), etc. See the SWATCH documentation for a complete detail of features and configuration options. In 2 months of use, this system has not only effectively deterred possible wily hackers, but also aided in the diagnosis of a multitude of network and client-server communication problems. We became aware of traffic entering our network by means that we didn’t even know existed: compromised network entities, internal mischief such as XXX web browsing, Napster activity and plenty of misuse of equipment (chat, day-trading, online gaming). Since we were able to use equipment that was considered obsolete and of little value, spend only $30 on software (the Redhat 6.2 CD) and the complete implementation took less than a week with no detrimental network impact, the system has more than paid for itself already.
7. Is that It?
Of course not. As new releases and fixes are available for each of the software components, the sensors and console will need updating. Use shell scripts and cronjobs to automate as much as the install/update process as possible as well as log rotation/archival. A thorough RTFM (Read The Fine Manual) session should be indulged upon for each of the software packages, as well as familiarization with the online references and download locations. As mentioned above, the optimal implementation would consist of a fully redundant/load balancing ID console. Choosing identical hardware for the ID sensors allows for rapid initial and subsequent deployment. Try to keep a backup sensor at the ready in case of a sensor outage and a master image on at least one hard drive at each location where a sensor is deployed. Choose sensor locations wisely, keeping in mind bandwidth limitations (~50mbps or less for maximum performance). Protect both sensors and console with ipchains (see the Linux Firewall HOWTO). Sign up for security mailing lists, especially those focused on the tools in use. Keeping up-to-date is an essential component of any security system; since open source software evolves so rapidly, even more emphasis should be placed upon maintaining current, stable releases.
Good luck and happy hacker hunting.
Stevens, W. Richard. TCP/IP Illustrated, Volume 1. Reading: Addison Wesley Longman, Inc, 1994.
Northcutt, Stephen and Novak, Judy and McLachlan, Donald. Network Intrusion Detection Second Edition. Indianapolis: New Riders Publishing, 2001.
Vision, Max. "Max Vision’s Whitehats." 2001 URL: http://www.whitehats.com (1 March 2001)
Vision, Max. "Re:Detecting exploits/shellcode." 16 June 2000. URL: http://archives.neohapsis.com/archives/ids/2000-q2/0157.html (30 March 2001)
Williamson, Glenn. "Re: Strange Scan." 18 December 2000. URL: http://www.securityfocus.com/frames/?content=/templates/ archive.pike%3Flist%3D75%26tid%3D151426%26end%3D2001-04-07%26threads%3D0%26start%3D2001-04-01%26 (2 March 2001).
Free Software Foundation, The. "The General Public License." Version 2. June 1991. URL: http://www.opensource.org/licenses/gpl-license.html (7 March 2001).
MITRE Corporation, The. "Common Vulnerabilities and Exposures." 2001. URL: http://cve.mitre.org (5 March 2001)
Xtremist. "Writing anti-IDS shellcode." URL: http://darknet.hq.alert.sk/papers/basicoverflows/stealthcode.txt (8 March 2001)
SecurityFocus.com. "BugTraq" 2001. URL: http://www.securityfocus.com (5 March 2001)
Open Source Initiative, The. "Open Source Initiative-Homepage." 2001. URL: http://www.opensource.org (8 March 2001).
Neohapsis, Inc. "Neohapsis Archives." 11 January 2001. URL: http://archives.neohapsis.com (12 March 2001)
Boran, Sean. "Hardening Red Hat Linux with Bastille." 1 April 2000. URL: http://www.securityportal.com/cover/coverstory20000501.html (1 March 2001).