Version .04 - January 6, 2000
There are a large number of variations of probes and attacks primarily directed against (presumably) SUN Solaris systems. In this chapter, we will discuss how to identify attacks that rely on access of the portmap program and that go directly to the service. Both are very common and quite dangerous as well. We will also discuss a bit about analysis of patterns in a large scale, information warfare style attack. We will ask you to look at data at a very sensitive time and try to imagine your job was to help make the call on what the world wide alert level should be. Finally we will revisit our friend nmap again, turns out a discussion of RPCs without a mention of nmap is a waste of time.
In a large number of attacks and probes, the signature is an access attempt against portmap. The first trace shown was detected using portsentry, a host based intrusion detection product with active defense. Portsentry uses an access list, the attacking computer in this case 172.20.20.1 will access TCP port 111, the expected location of portmap. Note that portsentry alerts to an attack and then in the second and third entry it reports on the action that it has taken, in this case it blocks the attacker's IP address (172.20.20.1) both with TCP Wrappers and also with the local ipchains firewall.
Active System Attack Alerts
=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dec 26 09:27:41 ext portsentry[12857]: attackalert: SYN/Normal scan from
host: 172.20.20.1to TCP port: 111
Dec 26 09:27:41 ext portsentry[12857]: attackalert: Host 172.20.20.1 has
been blocked via wrappers with string: "ALL: 172.20.20.1 : DENY"
Dec 26 09:27:41 ext portsentry[12857]: attackalert: Host 172.20.20.1 has
been blocked via dropped route using command: "/sbin/ipchains -I input -s
172.20.20.1 -j DENY -l"
rpcinfo -p | grep 32772
100024 1 udp 32772 status
100002 2 tcp 32772 rusersd
100002 3 tcp 32772 rusersd
# man rusersd
rpc.rusersd(1M) Maintenance Commands rpc.rusersd(1M)
NAME
rpc.rusersd, rusersd - network username server
SYNOPSIS
/usr/lib/netsvc/rusers/rpc.rusersd
DESCRIPTION
rpc.rusersd is a server that returns a list of users on
the host. The rpc.rusersd daemon may be started by inetd(1M)
or listen(1M).
03:47:34.116255 192.168.42.6.1763 > 10.1.86.0.111: S
467400212:467400212(0) win 32120 (DF) (ttl 47, id 4282)
4500 003c 10ba 4000 2f06 b889 c0a8 2a06
1001 5600 06e3 006f 1bdb f614 0000 0000
a002 7d78 73be 0000 0204 05b4 0402 080a
0deb ad2a 0000
03:47:34.116596 192.168.42.6.1764 > 10.1.86.1.111: S
463993015:463993015(0) win 32120 (DF) (ttl 47, id 4283)
4500 003c 10bb 4000 2f06 b887 c0a8 2a06
1001 5601 06e4 006f 1ba7 f8b7 0000 0000
a002 7d78 714d 0000 0204 05b4 0402 080a
0deb ad2a 0000
rpcinfo -p $1 >> $TMPFILE
From the script (running on attacker's (172.0.0.1) system
---------------------------------------------
# ./rpcscan.sh zappa
Let's find some exploitable services eh?
#
From syslog
-----------
Jan 3 15:03:59 zappa portmap[6790]: connect from 172.0.0.1 to dump(): request from
unauthorized host
Dec 27 15:14:25 marlin rpcbind: refused connect from 172.20.20.1 to dump()
Dec 27 15:14:40 marlin rpcbind: refused connect from 172.20.20.1 to dump()
Dec 27 15:14:43 marlin rpcbind: refused connect from 172.20.20.1 to dump()
Dec 28 15:47:03 milo rpcbind: refused connect from 172.20.20.1 to dump()
Dec 28 18:51:09 milo rpcbind: refused connect from 172.20.20.3 to dump()
|
So then, we surmise these traces targeted at dump() are reconnaissance as opposed to a direct exploit attack. In the trace below we have the good fortune of having two views of the same event from a system log (daemon.log) and also from a network log (tcpdump). Having multiple sensor views gives us a better picture of what is happening. We open with a trace grepped from a logfile called daemon.log. This should be pretty familiar to us by now.
daemon.log:Dec 27 15:20:35 zappa First the opening, we have the three way handshake complete with, SYN, SYN/ACK, Lone ACK. Please note that the port pairs here are 856 and 111, this could indicate the attacker is root (or a program running as root) on a Unix system.
15:20:35.513401 172.20.20.1.856 |
Fusing host and network logs One of the things that we learned over the Christmas holidays of the millennium turnover, was that intrusion detection state of practice is not yet advanced enough to allow the analyst to rely on network traffic alone. The process of using both system and network logs to determine an accurate battle picture is called fusion by the U.S. Department of Defense, we think it is both a good term and a great concept. Please take some time as you work though through each example to ask yourself is it a system log, or a network log. As you continue to examine logs and become familiar with their content, a wonderful thing happens, you can read them without thinking about the source. You just know it is a system log, or a network log and what that means to the battle picture, you are fusing! At the end of the chapter we will remind you to review the material you have covered and make your own determination on whether fusing is something you need to pursue in your analysis journey. |
Now the body of the connection, at this point a TCP connection is established and data can be passed. Defensively speaking, this is way too close for comfort for my taste. Now we will see data pushed from the prober (172.20.20.1) to the system (zappa). Note that zappa also sends data to the prober, if you don't want to keep track of the sequence numbers, the quick and dirty thing to do is look for the numbers in the parentheses (44) and so forth. This isn't just the lazy way to read traces, it can be crucial in a triage situation. Seasoned analysts in large facilities that come under attack may need to practice triage. As the attack traces come the analyst has a few seconds to evaluate them for their severity. The ones that get knocked down by the firewall are of interest in such a situation simply to know the kinds of attacks directed against your site and are just glanced at. A trace like the one we are working on may require additional consideration, the connection did get established, what happened, do we need to pull the system logs? Let's continue reading the trace together.
15:20:35.567136 172.20.20.1.856 > zappa.111: P
1:45(44) ack 1
win 16060 (DF) (ttl 55, id 30363)
15:20:35.725410 zappa.111 > 172.20.20.1.856: . ack 45
win 33536 (DF) (ttl 60, id 5373)
15:20:35.889546 zappa.111 > 172.20.20.1.856: P 1:33(32) ack 45
win 33580 (DF) (ttl 60, id 5374)
15:20:35.930580 172.20.20.1.856 > zappa.111: F
45:45(0) ack 33 win 16060 (ttl 55, id 30446)
15:20:35.931557 zappa.111 > 172.20.20.1.856: . ack 46
win 33580 (DF) (ttl 60, id 5375)
15:20:35.932534 zappa.111 > 172.20.20.1.856: F 33:33(0) ack 46
win 33580 (DF) (ttl 60, id 5376)
15:20:35.968683 172.20.20.1.856 > zappa.111: . ack 34
win 16060 (DF) (ttl 55, id 30461)
Before we move on to direct access of programs, did you notice the IP ID values in the body and close of the connection? Can we make any observations about our attacker? It seems reasonable that that our attacker's machine is pretty busy, because the numbers skip, zappa on the other hand seems fairly dedicated to serving the prober, perhaps this is an automated process probing a number of systems.
So far in this chapter we have been concerned with RPC attacks that use portmapper to locate the procedures they wish to attack. Attackers also go after RPC programs directly. Now we will take a look at this, starting with the classic attack signature, 32772 and moving on to some patterns that are still not understood at this time, but that certainly indicate an attack.
In the trace below, a secured portmapper detects an attempt against port 32772.
Jan 1 19:04:51 scylla tcplogd: port 32772
connection attempt from M-^DM-^Z^D@^A@[10.96.22.193]
rpcinfo -p | grep 32772
100024 1 udp 32772 status
100002 2 tcp 32772 rusersd
100002 3 tcp 32772 rusersd
Dec 26 09:29:43 nms1 rpcbind: refused connect
from 172.20.20.14 to getport(100099)
Dec 26 09:29:21 192.168.3.5 rpcbind: refused connect
from 172.20.20.14 to getport(1000)
Dec 26 09:29:26 192.168.3.5 rpcbind: refused connect
from 172.20.20.14 to getport(1000)
Dec 26 09:29:30 192.168.4.8 rpcbind: refused connect
from 172.20.20.14 to getport(100099)
crash:/etc/rc2.d# rpcinfo
program version netid address service owner
[ ... lots removed ... ]
100099 2 192.168.1.2.autofs - superuser
Dec 26 09:27:34 marlin rpcbind: refused connect
from 10.12.20.6 to getport(1000)
Dec 26 09:27:39 marlin rpcbind: refused connect
from 10.12.20.6 to getport(1000)
Twistah McLain was kind enough to email me an attack script that probes RPCs, a section of which is shown below:
#!/bin/sh
#all of these shiz are exploitable and i have 0 day for
#so i decided to make a rough scanner to sort through the rpcinfo krap
#[eCh0@LucidX.com]
TMPFILE=/tmp/.rpcscanfile
TOOLTALK=100083
STATUS=100024
AUTOFSD=100099
NISD=100300
MOUNTD=100005
PCNFSD=150001
REPORT=" was found on tcp port "
function scan() {
echo "Let's find some exploitable services eh?"
In the following trace, we see there are even more variations of this directed against systems all over the world.
Dec 30 17:28:04 babble rpcbind: refused connect
from 128.32.36.33 to getport(100099)
Dec 30 17:18:00 darkstar rpcbind: connect
from loopback(\000\000\346\365) to unset()
Dec 30 17:18:00 darkstar rpcbind: connect
from
Dec 30 17:27:47 darkstar rpcbind: refused connect
from 128.32.36.33 to getport(100099)
Dec 31 07:42:34 darkstar rpcbind: refused connect
from 128.8.176.40 to getport(100099)
"This is not a new technique. I most recently read a reference to it on page 232 of "Hacking Exposed," by Stuart McClure, Joel Scambray, and George Kurtz. (I heartily recommend this book -- http://www.hackingexposed.com) It discusses vulnerabilities in NFS. My comments are in brackets
'...never include the server's local IP address or localhost in the list of systems allowed to mount the file system. Older versions of the portmapper would allow attackers to proxy connections on behalf of the attackers. If the [attacker's] system were allowed to mount the exported file system [on a victim machine], attackers could send NFS packets to the target system's portmapper, which in return would forward the request to the localhost. This would make the request appear as if it were coming from a trusted host, and bypass any related access controls."This technique may be at work in your traces involving connections from "loopback.' "
So the bottom line, the loopback command may look odd, but once we know what it is used for, we are able to classify this a recon probe. Of course if a recon probe is successful, nine times out of ten a skilled attacker could use that information to launch a more devastating attack.
|
Today is January 6, 2000, by the time you read this book, we probably will have figured out what all of these patterns mean and have the analysis posted on the GIAC web page, after all the only one that is still unresolved today is the binary characters to getport(). However, we need to take you on a journey through the material as it looked on December 31, 1999. Right now, today, as you are reading this you have the advantage of all the analysis that has occurred since this was written and that will help you with unset() and friends, but it will not help you as much if you find yourself as an analysis in a firefight. So we will still be learning patterns but we will also learn, depth and breadth which are far more important.
Twas the week after Christmas 1999 and all through the net, there were several reasons to be concerned. CERT had released their denial of service advisory about tools like trinoo, tfn2000 and a warning that December 31 a large attack might be directed against web servers. There were claims everywhere about Y2K viruses, special viruses that would go off on the millennial turnover and hackers that were going to it us hard hiding in the noise of Y2K software problems. Finally there was Y2K, a lot of my friends were very concerned. That was the backdrop that caused us to begin the information monitoring and sharing effort that became known as the Global Incident Analysis Center. So now, take a second and put yourself in the lead incident handler seat for your organization and pretend these traces were coming into systems and networks that are your responsibility. You have already seen the attack information shown above and have read all the gloom and doom alerts and then you see the traces from the next section. As we like to say, severity is best evaluated from the point of view of the system owner! |
The role of the blitz in information warfare Whether you wish incapacitate your enemy, or simply need a diversion, the blitz is your friend in information operations. In football you can blitz against the run, or the pass, you can call it a blitz if you use one additional player to rush the passer or if you throw the kitchen sink at the opposing team and that illustrates magnitude and variation. You can conduct offensive information operations by using a large scale attack where you throw a large number of exploits at your opponent from a large number of source locations, and or make use of proxy tunnels. Size does matter, but this is not the scenario that most defenders fear, you can also throw a large number of attacks that they have never seen before. There was a time when we would have never believed that a large body of exploits could exist we had never prepared for, December 1999 taught us that wasn't true, we did see a huge number of attacks never before discussed. |
The trace below has the unset() attack we were just introduced to. In the unset() breakthrough sidebar we see that this activity can probably be attributed to reconnaissance, a showmount -e command or equivalent.
Dec 28 16:46:18 jeru rpcbind: connect
from loopback(\000\000\024\036) to unset()
Dec 28 16:46:18 jeru rpcbind: connect
from loopback(\000\000\024\037) to unset()
Dec 28 16:46:18 jeru rpcbind: connect
from loopback(\000\000\024) to unset()
|
The trace below brings up a new class of problems, the control question mark series to getport(). This attack has been attributed to the XXX, but at the time, we had never seen such a pattern. You can imagine our concern when you consider the number of attacks that had been detected worldwide on the week between Christmas and New Years (depth or magnitude) and the variety or breadth of these attacks.
Dec 29 05:35:46 dns1 rpcbind: refused connect from Even worse, the trace below shows this has been going on for some time and there is a getport(3000 | ) as well. The trace below shows several of the patterns.
Dec 12 17:58:08 robot rpcbind: refused connect |
unset() breakthrough One of the interesting things about the GIAC project is how we learn from one another. Here is a note I received January 2, 2000. Hi, I am not sure this is what you were looking for but it is Sunday morning, it is raining outside and I feel like reading a bit of C. So, according to the source from 4.4BSD and Wietse's secure portmap what we are seeing in the loopback calls to unset() is actually an attempt to unset an NFSD binding. At which point, coming from a localhost, it could well be something to do with a showmount -e but this is not entirely obvious as a showmount asks for an enumeration - what I'd do is atttach gdb to a showmount and see if it does a mount/umount sequence to try and get the mount list. Ciao, Arrigo Triulzi |
Dec 30 16:00:09 thumper portmap[21053]: connect
from IP.32 to callit(nsrstat): request from unauthorized host
Dec 30 16:00:12 thumper portmap[27908]: connect
from IP.32 to callit(nsrstat): request from unauthorized host
Dec 30 16:25:34 thumper portmap[18086]: connect
from IP.32 to callit(nsrstat): request from unauthorized host
Dec 30 16:25:37 thumper portmap[16721]: connect
from IP.32 to callit(nsrstat): request from unauthorized host
Dec 31 07:42:43 locust rpcbind: [ID 884469 daemon.warning]
refused connect from 128.8.176.40 to getport(1000<)
If it help any, we stayed at green when we did this for real, heck there are a lot of attempts but most of these are being defeated by host and network perimeter devices. Here is another trace.
Dec 26 15:17:28 6E:kestrel /usr/etc/portmap[150]:
rejected prog 100000 proc 3 call from 10.208.120.9
> uname -a
SunOS paganini 5.6 Generic_105181-03
sun4m sparc SUNW,SPARCstation-LX
> grep ttdbserverd /var/adm/messages
Dec 27 18:26:46 paganini /usr/dt/bin/rpc.ttdbserverd[3481]:
_Tt_file_system::findBestMountPoint --
max_match_entry is null, aborting...
Dec 27 18:26:46 paganini inetd[139]:/usr/dt/bin/rpc.ttdbserverd:
Segmentation Fault
Dec 27 18:26:56 paganini unix: rpc.ttdbserverd[3482]
attempt to execute pre on stack by uid 0
Oh nmap! The intrusion analyst must know the nmap signatures when it probes RPC ports. It is so widely used that we should ask ourselves whenever we see an RPC attempt, could this be nmap?
Let's start this part of our discussion with a view from the attacker's side of the table, here is a trace run by Mary Chaddock against a Solaris 5.5.1 system. Please tune your analysis eyes to port 111, 2049, and 32776, these TCP active ports are of particular interest to us.
Starting nmap V. 2.3BETA9 by Fyodor (fyodor@dhp.com, www.insecure.org/nmap/)
Interesting ports on sam.edu (192.168.8.96):
| Port | State | Protocol | Service |
| 21 | open | tcp | ftp |
| 22 | open | tcp | ssh |
| 25 | open | tcp | smtp |
| 80 | open | tcp | http |
| 111 | open | tcp | sunrpc |
| 139 | open | tcp | netbios-ssn |
| 389 | open | tcp | ldap |
| 515 | open | tcp | printer |
| 554 | open | tcp | rtsp |
| 2049 | open | tcp | nfs |
| 2766 | open | tcp | listen |
| 3306 | open | tcp | mysql |
| 5050 | open | tcp | mmcc |
| 8080 | open | tcp | http-proxy |
| 32776 | open | tcp | sometimes-rpc15 |
TCP Sequence Prediction: Class=random positive increments
Difficulty=39182 (Worthy challenge)
Remote operating system guess: Solaris 2.5, 2.5.1
Nmap run completed -- 1 IP address (1 host up) scanned in 6 seconds
|
sometimes-rpc
nmap has a table that is tweaked from time to time and may be the best source for interesting RPC ports. As Daniel Ayers pointed out, NMAP is shipped with a file, nmap-rpc, that lists the RPC programs and program numbers it knows about. (This file is attached - these are the RPC programs it will look for by default). If it can't locate that file anywhere, NMAP will use /etc/rpc.
# This was created by Vik Bajaj with help |
We also see that nmap is able to fingerprint the operating system without much trouble. Now the sometimes-rpc15 does deserve some explanation, but that is coming. First though, let's do one more trace of the same machine with a UDP focus.
Starting nmap V. 2.3BETA9 by Fyodor
(fyodor@dhp.com, www.insecure.org/nmap/)
Warning: No tcp ports found open on this machine,
OS detection will be MUCH less reliable
Interesting ports on sam.edu (192.168.8.96):
| Port | State | Protocol | Service |
| 111 | open | udp | sunrpc |
| 137 | open | udp | netbios-ns |
| 138 | open | udp | netbios-dgm |
| 514 | open | udp | syslog |
| 2049 | open | udp | nfs |
| 32771 | open | udp | sometimes-rpc6 |
Remote OS guesses: HP-UX B.11.00, HP-UX 11.00,
MacOS 7.5.5 - 8.6, MacOS 8 running on an LC 475,
MacOS 8.5, Solaris 2.3 - 2.4, Solaris 2.5, 2.5.1,
Solaris 2.6 - 2.7, Solaris 2.6 - 2.7 X86,
Solaris 2.6 - 2.7 with tcp_strong_iss=0,
Solaris 2.6 - 2.7 with tcp_strong_iss=2
Nmap run completed -- 1 IP address (1 host up) scanned in 1400 seconds
This time we see 111, 2049 and 32771 as ports of interest in terms of RPCs. As nmap warned, the TCP fingerprinting is not as good when you can't to TCP! Is the nmap signature obvious on the network? Very, for one thing a port scan is big, big, big, so we will just look at a couple excerpts. In the first trace we will just take look at a small chunk of closed ports.
18:50:52.831580 waldo.edu.47862 > sam.edu.936: udp 0
18:50:52.831686 waldo.edu.47862 > sam.edu.937: udp 0
18:50:52.831768 waldo.edu.47862 > sam.edu.327: udp 0
18:50:52.831849 waldo.edu.47862 > sam.edu.635: udp 0
18:50:52.831928 waldo.edu.47862 > sam.edu.554: udp 0
18:50:52.832007 waldo.edu.47862 > sam.edu.321: udp 0
18:50:52.832084 waldo.edu.47862 > sam.edu.1600: udp 0
Now let's move on the TCP. We aren't going to dwell on the options which as we write this do help serve as a signature for namp, but again, you can run it yourself and see what the footprint looks like on your network. We will look at two cases, one where the port is not open, the second where it is.
In this case we have a TCPdump trace of a probe by nmap to a closed port, 32771. We see two packets the active open, the SYN is set, the ISN is 2981802919. The response from sam is a reset and the ISN is incremented by one.
18:50:08.064279 waldo.edu.61345 > sam.edu.32771: S
2981802919:2981802919(0) win 8192 <mss 1460,nop,wscale
0,nop,nop,timestamp 13486573 0> (DF)
18:50:08.066649 sam.edu.32771 > waldo.edu.61345: R
0:0(0) ack 2981802920 win 0 (DF)
18:50:08.238249 waldo.acu.edu.61721 > sam.acu.edu.32776: S
3006255052:3006255052(0) win 8192 <mss 1460,nop,wscale
0,nop,nop,timestamp 13486574 0> (DF)
18:50:08.254079 sam.acu.edu.32776 > waldo.acu.edu.61721: S
3403285064:3403285064(0) ack
3006255053 win 8760 <mss 1460> (DF)
18:50:08.254167 waldo.acu.edu.61721 > sam.acu.edu.32776:
. ack 1 win 8760 (DF)
18:50:08.272890 waldo.acu.edu.61721 > sam.acu.edu.32776:
R 1:1(0) ack 1 win 8760 (DF)
