Background and Introduction
The IDS world has finally gotten its arms around switched LAN's, but what about networks with redundant components? How do you know you're seeing all the attacks?
This paper examines common forms of network redundancy, strategies for placing network IDS probes in redundant networks, and the effectiveness of those strategies.
The goal here is to allow a single IDS probe to see all the traffic associated with a particular attack from one host to another regardless of how many packets the attack takes, how many network-level "sessions" the attack takes, or what network paths the packets traverse.
Redundant Network Elements
The following table gives a taxonomy of redundant network elements we'll consider in this paper:
How to
place IDS sensor in redundant networks?
Chris Calabrese
Background and Introduction
The IDS world has finally gotten its arms around switched LAN's, but
what about networks with redundant components? How do you know you're
seeing all the attacks?
This paper examines common forms of network redundancy, strategies for
placing network IDS probes in redundant networks, and the effectiveness
of those strategies.
The goal here is to allow a single IDS probe to see all the traffic
associated with a particular attack from one host to another regardless
of how many packets the attack takes, how many network-level "sessions"
the attack takes, or what network paths the packets traverse..
Redundant Network Elements
The following table gives a taxonomy of redundant network elements
we'll consider in this paper:
Note that these elements are orthogonal and may be arbitrarily combined.
However, it would be unusual to combine two elements from the same row,
such as NIC failover and route-based NIC multi-pathing, since they're
aimed at the same purpose.
The rest of this section serves as a brief introduction to how these
elements operate, how they're useful, and, and their challenges for IDS
sensor placement.
NIC failover
With NIC failover, an individual network
node (typically a server or router) has multiple links to its LAN (or LAN's)
through redundant NIC's that allow failover of the system's network address(es)
from a primary NIC to a secondary NIC. This can ensure that the system continues
operating in the face of a NIC failure.
In this scenario, failover software on the network node itself detects
that the primary NIC is inoperative, de-configures it from use at the OS
level, and configures the secondary NIC to take its place. In more
sophisticated implementations, both the IP address and MAC address will
fail over, allowing the secondary NIC to start talking on the network
immediately. In less sophisticated implementations only the IP address
will fail over, causing delays as ARP caches time out.
If we assume that it's highly unlikely for a NIC to fail in the middle
of an attack, then the IDS placement challenge is to ensure that the
traffic to/from both NIC's is covered, but not necessarily by the same
IDS probe.
NIC multi-pathing through route advertisements
With
NIC multi-pathing in general, an individual network node (typically a server
or router) has multiple links to its LAN (or LAN's) through redundant NIC's
operating in parallel. This allows higher network throughput compared to
NIC failover.
With multi-pathing through route advertisements, the system has one IP
address per NIC and advertises routes to its various IP addresses
through each of its other IP addresses. Solaris 8, for example, does
this by responding appropriately to Router Discovery Protocol requests[
1 ].
The IDS challenge with multi-pathing through route advertisements is for
one IDS probe to see all the network traffic regardless of which NIC the
traffic is going to. Otherwise the IDS would have to do complex
multi-probe correlations to construct a single attack.
Multi-pathing through switch link-aggregation advertisements
This is very similar to the multi-pathing through route
advertisements, but here the system advertises paths to itself at the
switching level instead of the routing level. The HP-UX implementation,
for example, uses Cisco's Fast EtherChannel protocol to tell the switch
how to send packets for the system's MAC address(es)[
2].
The IDS challenge is the same as for multi-pathing through route
advertisements.
Failover clustering
With failover clustering,
network nodes (servers, routers, firewalls, etc.) can take over each other's
network addresses when the primary address holder fails. This is similar
to the failover NIC scenario discussed above, but with the failover happening
at the entire-machine level rather than at the NIC level. As with NIC failover,
just the IP address or the IP and MAC addresses can fail over depending
on the sophistication of the software. This can ensure that a key service
to continues operating in the face of a server failure.
If we assume that it's highly unlikely for a system to fail in the
middle of an attack unless the result of the attack, then the IDS
placement challenge is to ensure that the traffic to/from both systems
is covered, but not that traffic to all cluster members must be covered
by a single IDS probe.
Independent IP clustering
With independent IP clustering,
each system in the cluster has its own unique IP address, and the client
applications must decide how to distribute the load among the different
servers. This is the basis of DNS clustering (the client choses among
multiple DNS servers that are authoritative for the domain) and SMTP clustering
(the client choses among multiple SMTP servers that have MX records in the
domain)[ 3 ].
The IDS placement challenge with independent IP clustering is to ensure
that a single probe sees all the traffic for a particular
source/destination address pair. This may require multiple probes if
the systems are geographically disparate.
Shared IP clustering
With shared IP clustering,
an external load balancer is used to distribute requests sent to a shared/virtual
network address to multiple servers. Each server in the cluster has
their own unique/real IP address, and traffic between the load balancer
and the real servers reflects these addresses rather than the shared address.
This is how most popular web server load balancing systems work.
The IDS placement challenge with shared IP clustering is to ensure that
a single probe sees all the traffic for a particular source/destination
address pair (i.e., all traffic for a single attack). However, if we
assume that the load balancer keeps all traffic from a particular source
going to a particular destination as long as that destination is alive,
and if we further assume that it's highly unlikely for a server to go
down during an attack except as a result of the attack, than this
collapses to all the traffic for a particular source regardless of which
server the load balancer decided to send the traffic to.
Multiple switches
Merely having multiple switches
handling the same network segment does not itself make the switches redundant
or contribute to higher network availability. But multiple switches
can be combined with redundant NIC's, failover routers, etc. so that the
network will stay up in the event of a single switch failure.
The IDS placement challenge is to ensure that all data from all switches
is seen by the IDS probes.
Parallel routes
In this scenario, a network may have redundant links internally
(i.e., it is not a tree/hierarchy) or have multiple links to the outside
world (i.e., multiple ISP links). This is typically used to ensure that
network connectivity can survive a single link being down and also to
increase network throughput.
The IDS placement challenge is to ensure that a single probe sees all
the traffic that may flow between a particular source/destination pair
so that multi-probe correlation is not necessary to construct a single
attack.
Load-balanced firewalls
This is similar to the shared IP clustering scenario, but here the
load balancers are placed both in front of and behind the firewall farm.
This is used both to ensure that network connectivity will remain in the
face of a firewall failure and to increase overall throughput through
the firewall farm.
The IDS placement challenge is to ensure that a single probe sees all
the traffic associated with a particular attack, and therefore a
particular IP source/destination pair. However, similarly to the shared
IP clustering scenario, load balancers keep all traffic for a particular
source/destination pair going through the same firewall as long as the
firewall is alive, and if we further assume that it's highly unlikely
for a firewall to go down during an attack except as a result of the
attack, than a single attack will be tied to a single firewall.
IDS Placement Strategies
In this section we'll consider the following placement strategies:
- Single-segment
IDS probe placed outside area of redundancy
- Multiple
single-segment probes
- Multi-segment
probe
- Multiple multi-segment
probes
Single-segment IDS probe placed outside area of redundancy
A single IDS probe is used, but carefully placed to avoid the
problems introduced by redundant network elements. The probe could be a
general-purpose system running a software IDS, a purpose-built IDS
appliance, or a special purpose IDS card for a network switch such as
Cisco's Catalyst 6000 IDS Module[
4s ] or Intrusion, Inc.'s SecureCom 6000 series[
5 ].
The challenge is finding a location where the IDS probe can see all the
traffic. The table below shows where a probe might be located when used
with different redundant network elements. Remember when reading the
table, however, that the elements are orthogonal, so a location can only
be used if it works for all the redundant elements in the network. For
example, a switch SPAN port only works if you have a single switch or if
the switches you have support multi-switch spanning.
Redundant Network Element
|
IDS Probe Location
|
| Failover NIC's |
Switch SPAN port
|
| NIC multi-pathing |
Switch SPAN port
|
Failover cluster
|
Switch SPAN port
|
Independent IP cluster
|
Switch SPAN port, if not geographically
disparate
|
Shared IP cluster
|
In front of load balancer
|
Multiple switches
|
Multi-switch SPAN port
|
Parallel routes
|
Whatever part of the network is not parallel, if any
|
Load-balanced firewalls
|
In front of or behind entire cluster
|
Aswitch SPAN port is a network switch port setup so that that the
switch copies all packets traveling through the swich (or a particular
VLAN or set of VLAN's) to that port. This is sometimes referred to as
port mirroring, though technically that refers to copying packets from
one particular port to another, rather than copying all the packets from
all the ports.
A multi-switch SPAN port refers to a SPAN port that sees data from
multiple switches. In this way a single IDS probe can see span traffic
from multiple switches. The poor man's version of this is to connect the
SPAN port of one switch into another switch, but this can be cumbersome
and cause performance problems and delays. Some high end switches such
as the Cisco Catalyst 6000 family of switches have native support for
inter-switch spanning over the normal inter-switch backbone connections[
6 ].
Multiple single-segment probes
Next up the food chain is using multiple single-segment probes to
cover multiple network segments. This gives us more flexibility to
cover situations where failover network elements are used in conjunction
with other redundant elements such as multiple switches. However, this
technique still can not address issues where the traffic from a single
attack may be sent along multiple paths that can't be spanned, such as
multi-path NIC's on multiple switches that don't support multi-switch
spanning. Having multiple probes also increases acquisition and
management costs.
The following table summaries where multiple single-segment probes can
offer an improvement over a single single-segment probe.
Redundant Network Element
|
Advantages/Disadvantages of Multiple Probes vs. Single
Probe
|
| Failover NIC's |
Improves flexibility to work with other redundant elements
|
| NIC multi-pathing |
No difference
|
| Failover cluster |
Improves flexibility to work with other redundant elements
|
Independent IP cluster
|
Improves performance, ability to handle geographic
diversity (locate by individual servers rather than in front of cluster)
|
Shared IP cluster
|
Improves performance (locate by individual servers
rather than in front of cluster)
|
Multiple switches
|
No difference
|
Parallel routes
|
No difference
|
Load-balanced firewalls
|
Improves performance (locate by individual firewalls
rather than in front of / behind cluster)
|
Multi-segment
probe
An IDS probe that can listen on multiple network segments can cover
arbitrarily complex redundant network scenarios as long as the whole
network is in the same building. And less probes means lower management
costs. But should we always jump to this design if a single
single-segment IDS probe won't cut it, or are there situations where
multiple single-segment probes would be more effective? Or even a
combination of the single- and multi-segment probes. The issues are
going to be:
- What is the acquisition cost vs. multiple single-segment probes?
- Can a single complex probe handle the traffic levels?
The answers to these questions depend greatly on exactly how the
probe is constructed. We have two options:
- Probe with multiple NIC's
- Probe connected to a specialized switch
Let's look at these two options in more detail.
Probe with multiple NIC's
Not all NIDS software supports multiple NIC's. For example, SNORT
can not look at more than one data stream at a time[
7< ]. Only higher-end versions other systems may support this
feature, especially on appliance-based probes, and the added cost may
not be justified if other high-end features (such as probe
performance) aren't required. Regarding performance, many IDS
technologies are not able to make use of SMP technology and therefore
are limited to roughly 100Mb/s total throughput[
7]. Even systems that support SMP may have very high costs
relative to performance due to non-linear scaling with the number of
processors.
Probe connected to specialized switch
It is possible to connect IDS systems that do not support multiple
NIC's to a specialized switch such as the Top Layer AppSwitch that
feeds the IDS probe from multiple ports on the switch[
8 ]. The cost of a Top Layer AppSwitch starts at around $10,000
at the time of this writing[ 9 ],
so again, this may not be a cost effective solution. However, the
AppSwitch is also capable of feeding multiple IDS probes and making
sure that each network flow goes to one and only one probe, so this
may be an excellent choice if very high performance is necessary.
To put this in our familiar format:
Redundant Network Element
|
Advantages/Disadvantages of Multi-Segment Probe vs.
Multiple Single-Segment Probes
|
| Failover NIC's |
No difference
|
| NIC multi-pathing |
Increases effectiveness/flexibility
|
| Failover cluster |
No difference
|
Independent IP cluster
|
Decreases performance, ability to handle geographic
diversity
|
Shared IP cluster
|
Decreases performance
|
Multiple switches
|
Increases effectiveness/flexibility
|
| Parallel routes |
Increases effectiveness/flexibility
|
Load-balanced firewalls
|
Decreases performance
|
Multiple multi-segment probes
Same advantages of performance as multiple single probes, but with
the flexibility of multi-segment probes.
Redundant Network Element
|
Advantages/Disadvantages of Multiple Multi-Segment
Probes vs. One Multi-Segment Probe
|
| Failover NIC's |
No difference
|
| NIC multi-pathing |
No difference
|
| Failover cluster |
No difference
|
Independent IP cluster
|
Increases performance, ability to handle geographic diversity
|
Shared IP cluster
|
Increases performance
|
| Multiple switches |
No difference
|
| Parallel routes |
No difference
|
Load-balanced firewalls
|
Increases performance |
IDS Probe "Sweet Spots"
Application Redundancy
(Clustering)
|
System Redundancy
|
Network Redundancy
|
Multiple Switches (no multi-switch
port SPANning)
|
Multiple Switches and Multiple Links
/ Segments
|
Multiple Switches, Multiple Links
/ Segments, and Firewall Load Balancer (high data rate)
|
No Clustering
or Failover Clustering
|
Failover NIC's
|
Simple probe |
Multiple simple probes
|
Multiple simple probes
|
NIC multi-pathing
|
Multi-segment probe
|
Multi-segment probe
|
Multiple multi-segment probes
|
Geographically
Disparate Independent IP Cluster
|
Failover NIC's
|
Multiple simple probes
|
Multiple simple probes
|
Multiple simple probes
|
NIC multi-pathing
|
Multiple multi-segment probes
|
Multiple multi-segment probes
|
Multiple multi-segment probes
|
Shared IP
Cluster (high data rate)
|
Failover NIC's
|
Simple probe |
Multiple simple probes
|
Multiple simple probes
|
NIC multi-pathing
|
Multiple multi-segment probes
|
Multiple multi-segment probes
|
Multiple multi-segment probes
|
Conclusions
No matter what kind of bizarre network architecture you have, it's
always possible to find some way of monitoring it with a network IDS.
However more complex networks may require multiple single-segment
probes, a multi-segment probe, or even multiple multi-segment probes.
References
- Mark Garner, "IP Network Multipathing," Sun
BulePrints OnLine, February 2001. Available from
http://www.sun.com/blueprints/0201/Multipathing.pdf .
- Hewlett Packard, " Cisco
Systems and hp fast etherchannel
and auto-port aggregation software ." Available from
http://www.hp.com/products1/unixserverconnectivity/infolib/cisco.html
.
- Paul Albitz and Cricket Liu, DNS and
BIND, 4th Edition , O'Reilly and Associates, 2001.
- Cisco Systems, "Catalyst 6000 Intrusion
Detection System Module Data Sheet." Available from
http://www.cisco.com/warp/public/cc/pd/si/casi/ca6000/prodlit/6kids_ds.htm
.
- Intrusion.Com, "SecureCom Chassis
Overview." Available from
http://www.intrusion.com/products/productcategory.asp?lngCatId=21
.
- Cisco Systems, "Configuring the Catalyst
Switched Port Analyzer (SPAN) Feature." Available from
http://www.cisco.com/warp/public/473/41.html .
- Martin Roesch, comments made during SNORT
BoF at SANS Network Security 2001 Conference, October 2001.
- Gary Kessler, "IDS-in-Depth: Top Layer's
AppSwitch filters a copy of traffic flows to downstream IDSeS,"
Information Security Magazine, August 2001.
- Top Layer Networks, private
correspondence, November 2001.
|
|
