[Editor's note: Due to changes in the Linux file system drivers, Hal has posted an update to this post at https://blogs.sans.org/computer-forensics/2009/10/05/mounting-images-using-alternate-superblocks-follow-up/.]
Mounting Unix file system images is a common investigative technique, because it allows examiners to use standard file system tools (e.g., find, grep) to look for evidence. However, sometimes you run into difficulties:
# mount -o loop,ro dev_sda2.dd /mnt
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
So what's going on here? Well, let's try the dmesg command suggested in the above error message:
# dmesg | tail [...] [565633.964688] EXT3-fs: INFO: recovery required on readonly filesystem. [565633.964698] EXT3-fs: write access unavailable, cannot proceed.
Actually, in this case the "file" command can provide us with more succinct information:
# file dev_sda2.dd dev_sda2.dd: Linux rev 1.0 ext3 filesystem data (needs journal recovery)
See the "needs journal recovery" bit at the end? Looks like the image comes from a machine that was shut down ungracefully, and the ext3 file system driver wants to sync the operations in the file system journal before allowing the file system to be mounted.
Now we could mount the image read-write and allow the journal to be flushed, but the concern is that we'd be damaging the forensic integrity of the image- to say nothing of losing actual evidence. In fact, the file system in the image is probably in a decent enough state that we could actually mount it, if only there were some way to coerce the file system drivers into not requiring a journal recovery. Needless to say, I wouldn't be writing this article unless there was a way to do this...
The Alternate Superblock Trick
As most of you are aware, Unix file system metadata is kept in the superblock at the beginning of the file system image. The superblock stores information like file system state (clean or dirty), last mounted time and location, and pointers to the root of the file system, free block and inode lists, and the file system journal (if any). For redundancy, the primary superblock is replicated throughout the file system. That way if you lose the primary superblock for some reason, you can often recover the file system using one of the alternates, depending on the nature of the corruption.
However, the data in the primary superblock is not perfectly replicated to the alternates. Most important for our purposes, the file system clean/dirty information that's causing the ext3 file system drivers to require a journal recovery is not flagged in the alternate superblocks. The upshot is that if we tell the mount command to use an alternate superblock, the OS will allow us to mount the file system image without requiring a journal recovery. The hard part is figuring out where the alternate superblocks live and then setting options for the mount command appropriately.
Mounting With Alternate Superblocks
The first step is to discover the location of the alternate superblocks. The trick is to use mkfs with the "-n" option. As with many other Unix commands, "-n" means "show me what you would do, but don't actually do it":
# mkfs -n dev_sda2.dd mke2fs 1.41.3 (12-Oct-2008) dev_sda2.dd is not a block special device. Proceed anyway? (y,n) y Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 58880 inodes, 235516 blocks 11775 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=243269632 8 block groups 32768 blocks per group, 32768 fragments per group 7360 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376
There are two important pieces of information in the above output. The first is obviously the list of alternate superblock locations at the end of the output. But we also need the block size information from earlier in the output.
Why is the block size relevant? It turns out that the mount command wants you to specify the alternate superblock location in terms of 1K offsets. But mkfs is reporting the alternate superblock addresses in terms of 4K blocks, so we'll need to multiply the values reported by mkfs by four in order to produce addresses we can feed into the mount command.
You can use any of the alternate superblock addresses you want- let's just use the first one. The offset we need to feed into the mount command then is 32768*4 = 131072. The mount option for specifying an alternate superblock is "sb=":
# mount -o loop,ro,sb=131072 dev_sda2.dd /mnt # ls /mnt bin dev home lib mnt proc sbin usr boot etc initrd lost+found opt root tmp var
Excellent! We can now view the contents of our file system image as a mounted file system.
Beyond Forensics
Of course, outside of the forensic context, being able to recover file systems using alternate superblocks is incredibly useful in case of file system corruption. While I hope none of you are ever in a situation where you need to do this, it can make you look godlike in the eyes of your co-workers and management. So keep this arrow in your quiver for emergencies.
Hal Pomeranz is an independent IT/Security Consultant and a SANS Institute Faculty Fellow. Rumors of his godlike powers are largely exaggerated.