First, for those of you who don't know, let me explain how HSTS works. HSTS is a HTTP header which a web server can send to tell a client that they should not accept unencrypted communications from that domain for a specified period of time. Developers can also preload their websites so that the browser knows that it should use HTTPS for its first communication.
strict-transport-security: max-age=31536000; includeSubDomains; preloadExample HSTS header
However, for this to work it must be writing to disk somewhere, and that means forensic artefacts! So I set out hunting for them and very quickly had the usual series of revelations that come with a project like this; "This seems really easy, why has no one done it before" "Oh... that's why...".
Before I go any further, I should note something very important. Like any browser cache artefact, a HSTS database record does not prove that the user deliberately browsed to that website, simply that the browser interacted with it.
Firefox
Let's start with Firefox, given that it's the simplest of the browsers to analyse. Firefox writes its HSTS database to a file called SiteSecurityServiceState.txt within the user's Firefox profile (%APPDATA%\Mozilla\Firefox\Profiles\ on Windows), but unlike most Firefox artefacts, it's not an SQLite file, but a plain text tab-separated table.
<pre><code>blog.daniel-milnes.uk:HSTS0182071604684330099,1,1,2<br></code></pre>Example entry
Let's break this down:
blog.daniel-milnes.uk - The domain in question.
:HSTS - This file is also used to store HPKP records, so this distinguishes the record as HSTS.
0 - The number of visits. Note: In my testing I found the behaviour of this field to be very unreliable, so I would caution against treating it as forensically sound.
18207 - The number of days since the Unix Epoch that the page was last accessed.
1604684330099 - The number of milliseconds after the Unix Epoch that this record expires.
1 - The Security Policy State. 0 meaning unset, 1 meaning set, 2 meaning knockout, and 3 meaning negative.
1 - Should subdomains be included? 1 means yes, 0 means no.
2 - The Firefox source code calls this source, but in my testing I was never able to get it to produce any value other than 2, including sites with and without preload.
Firefox does not consider this file to be history, so clearing history will not remove it, but it does consider it a Site Preference.
Clean up the HSTS databaseThe database is not written until Firefox is closed, meaning that a live disk capture may prove incomplete. This also applies to Tor browser, as it is based on the Firefox source code.
Chrome
Google Chrome proved to be a much harder beast to tame when it came to actually finding where the HSTS database is on disk, but after resorting to the tried and true DFIR method of crossing your fingers and using diff, I eventually found %LOCALAPPDATA%\Google\Chrome\User Data\TransportSecurity, a JSON file (despite the lack of the extension) containing the database.
<pre><code>"+2oHxdIbjeDrXH6buN8LtFwdxx7XuvmXd+B47y9TQIM=": {<br> "expiry": 1599899783.19529,<br> "mode": "force-https",<br> "sts_include_subdomains": false,<br> "sts_observed": 1568363783.195293<br>}</code></pre>Example entry
That doesn't exactly look like a domain, does it? Well I went digging in the Chromium source, and my heart sank when I saw this:
// This inverts |HashedDomainToExternalString|, above. It turns an external
// string (from a JSON file) into an internal (binary) string.
std::string ExternalStringToHashedDomain(const std::string& external) {
std::string out;
if (!base::Base64Decode(external, &out) ||
out.size() != crypto::kSHA256Length) {
return std::string();
}
return out;
}Yep. Chrome stores in the format Domain -> Replace . with hex showing distance to next . -> Add null terminator -> SHA256 -> Base64, meaning that the process would go blog.daniel-milnes.uk -> \x04blog\x0Ddaniel-milnes\x02uk -> \x04blog\x0Ddaniel-milnes\x02uk\x00 -> Sha256 -> Base64. I'm assuming this was done to help with privacy concerns, but it really makes forensics a nightmare. Nevertheless, there is still some value here if you're trying to prove that a suspect's browser visited a specific site, but you won't be able to dump out a list like you can with Firefox.
Beyond that, the other fields are laid out like so:
expiry - The Unix timestamp of the record's expiry.
mode - For HSTS this will say force-https.
sts_include_subdomains - Should subdomains be included?
sts_observed - The Unix timestamp when the resource was last observed.
Clearly, Google decided to kick forensicators when they were down at this point, because unlike Firefox where the option to delete the HSTS database is not ticked by default, Google not only bundle HSTS in with cache, but it ticks the option by default. That means that someone clearing up after themselves would have to fairly intentionally leave behind the artefact.
The only nice thing about Chrome is that the developers made the research marginally easier by creating a developer page where you can register HSTS domains chrome://net-internals/#hsts.
This is true for all browsers built on the Chromium project, including Chrome, Edge Dev, and Opera. Like Firefox, the file is written when the program is closed.
HSTS Parser
So none of that seems particularly easy to quickly analyse, but fortunately, I've hacked together some low-quality Python to help with this problem! HSTS Parser is now available on GitHub, and it can process Firefox and Chrome HSTS databases! It'll even give you a nice ASCII table to look at everything in.
Example Firefox outputWhilst I've not broken SHA256, you wouldn't be hearing about that for the first time here, I have added support for a wordlist when processing Chrome hashes. This means that you can feed in a list of domains you like to know if were contacted and it will hash them and try to match them to the list.
Example Chrome outputYou can get HSTS Parser on my GitHub here today!
Hopefully that gives you a good insight into the forensic applications of HSTS, but if you've got any questions or suggestions, feel free to drop me an email at daniel@daniel-milnes.uk!