Understanding EXT4 (Part 4): Demolition Derby

Hal Pomeranz, Deer Run Associates

In Part 3 of this series we looked at the EXT4 extent tree structure for dealing with very large or very fragmented files- basically any situation where you need more than the four extent structures available in the inode. Go back and read that part now if you haven't already, because what I'm going to be talking about here follows directly from that discussion.

Extent Structures After Deletion

I got curious about what would happen when I deleted my /var/log/messages file. How does the inode change? What happens to block 131090, which holds my extent tree structure? Well, there's really only one way to find out: I deleted the file... carefully so I didn't lose any logging data. In fact, I didn't just delete the file; I used "shred -u /var/log/messages" to overwrite the data blocks with nulls before unlinking the file. Once the file had been purged, I dumped out the both inode associated with the file as well as block 131090 and took a look at them in my hex editor.

Here's the inode:

Similar to our observations in Part 1, deleting the file causes the file size in bytes 4-7 to be set to zero, and the number of extents in the extent header structure to be marked as zero as well. The deletion process also sets the "depth of tree" field back to zero.

But what's really interesting to me is that the rest of the extent structures are intact! In Part 1 of this series, when we were dealing with a small file that only had a single extent structure in the inode, deleting that file caused the extent structure to be zeroed out. But here, the extent index structure is left alone- so I can actually read the block number of the data block that holds the rest of the extent tree. Furthermore, the old structures for extents 2-4- which were left in the inode even after the extent tree was created (see Part 3)- are still there as well. These tell me where chunks of the file can be found on disk, assuming the blocks haven't been re-used or clobbered by a tool like shred.

Things get even weirder when you look at what's left in block 131090:

If you look at the extent header structure at the start of the block, the "number of extents" field has been set to zero (everything else remains the same). What's really odd to me, however, is that the developers went to the trouble to null out much of the data in the extent structures in the block, but they left the logical block offsets alone (in fact, I think this also happened with the extent structure in the deleted inode from Part 1- it's just that the logical block offset there was zero anyway).

Seriously, WTF? Why go to all the trouble of zeroing out the last 8 bytes of each structure only? Why not just clobber the entire block, extent header and all? Obviously somebody thought it was important to do things this way, but I can't for the life of me understand why. Frankly, the treatment of both the inode and the extent tree structures in the data blocks upon deletion feels like a bug to me, rather than a feature.

Now maybe it's not a huge deal, but there is some forensic information I can glean from this residue. First, I can tell that there were six extents in the original file. Second, I know that the first four extents each contained one block- just subtract the starting point of extent <n> from the starting point of extent <n+1>. Similarly, we can see that extent 5 contained 124 blocks. The length of the last extent is unknown, given only the residual data. And, of course, we have the unaltered versions of extents 2-4 that were left behind in the inode itself, which includes the starting block addresses and lengths of these extents.

File Timestamps

We talked about timestamps in EXT4 back in Part 2 of this series. But we never actually looked at what happened to the timestamp values when a file gets deleted. Let's remedy that.

Here's our deleted inode with the timestamp fields marked out in color:

You'll notice that the mtime seconds (bytes 16-19, yellow) and ctime seconds (bytes 12-15, purple) are now set to the same value, which is the time the file was deleted. The fractional seconds in the "extra" fields for each of these timestamps are different- again I'm left wondering why the developers bothered.

The creation time value is entirely unchanged (compare with the view of the inode in Part 3 if you don't believe me), so the deleted inode will tell you when the file that last used that inode was created in the volume. The atime value is not affected by the delete operation either, though in this case other operations on the system have updated the atime value since we looked at it in Part 3.

EXT file systems actually do track an additional "deletion time" timestamp- it's bytes 20-23, marked in orange in the picture above. Frankly, deletion time has never been very useful because the mtime and ctime values end up getting set to the same value as dtime anyway. You can tell the EXT4 developers didn't care too much about dtime, because they didn't bother creating room for a "dtime extra" field in the upper 128 bytes of the inode like they did for the MAC timestamps. That also means you're going to have a "dtime rollover" problem in year 2106, while the other timestamps are good until 2514 (refer to the discussion in Part 2 of this series).

Wrapping Up

At this point we've covered a lot of EXT4, and probably in much more detail than you'll need to do your job on a daily basis. But if you're curious about anything else related to the file system, put your questions down in the comments and I'll respond- perhaps even write additional installments in this series.

I'm already starting to see EXT4 file systems "in the wild" on cases that I'm working. And the adoption of EXT4 as the file system for Android 2.3 and later means there's going to be even more pressure for vendors of forensic tools to step up and provide better support for this file system. As of this writing, DFF is the only forensic tool that I'm aware of that even claims to have support for EXT4.

Hal Pomeranz is an Independent IT/Security Consultant, a SANS Institute Faculty Fellow, and a GCFA. This series has been a thinly-veiled attempt to embarrass the developers of various forensic tools into supporting EXT4- mostly so that Hal can avoid doing the development work himself.