The Verizon Data Breach Report has consistently said, over the years, passwords are a big part of breach compromises. Dr. Lori Cranor, and her team, at CMU has done extensive research on how to choose the best password policies verses usability. In addition, Alison Nixon's research describes techniques to determine valid password of an organization you are not a part of ("Vetting Leaks Finding the Truth when the Adversary Lies"). What about passwords leaked in the organization you are defending? This post will be about such a scenario.
According to former Deputy Director, of The Center for The Studies of Intelligence, Ms. Carmen Medina says "analysis in essence is putting things correctly into categories" ? "insight is when you come up with a new category scheme that offers more explanatory power than the one you had" (18:35 min).
In essence, your hypothesis filters the data to find value and you need to constantly re-evaluate your hypothesis.
Early on in my IR forensic career, I learned the hard way, never assume, choose the hypothesis with the least assumptions. Know what your biases are, be careful to consistently keep them in check and never out rule other options.
So how can we check a password dump on the internet containing credentials purporting to be from our organization? Simple, does the password dump match our password policy? Let's put the data in more specific buckets and improve out threat intelligence process.
The following script will ask a user to input their password policy settings (length, minimum capital letters, etc.) and compare it to the password dump file, outputting a percentage of invalid passwords. The script saves time and provides a better category scheme for analysis because it put the data into "better buckets". A future improvement could be to script out the username prior to running he diff. The script allows the analyst to apply threat intelligence in context, making it practical. GitHub.
P.S. I've taken SANS 578 Cyber Threat Intelligence (beta) I highly recommend it.
# GitHub https://github.com/johnfranolich/threatintelligence
# PW Dump Verifier
# Jamie Gambetta and John Franolich