This three-part blog series is jointly authored by Sean O’Connor, co-author of SANS FOR589: Cybercrime Intelligence, and Freddy Murstad, PhD. candidate researching traditional intelligence methodology application into cyber threat intelligence (CTI). The blog series will highlight the benefit of using the Admiralty System when assessing cybercriminals and their claims of data breaches within cybercrime underground communities. In part 1, we will focus on the background and essential elements of the Admiralty System as well as highlighting some of the benefits and drawbacks of using the system. One of the focal points of FOR589 is learning to filter out the noise in the ‘underground’ and bring structure to an otherwise unstructured flow of data and sources. In part 2, we will evaluate cybercriminal sources and claims and discuss how to align them with this system.
Picture this: You’re drowning in alerts, your open-source intelligence (OSINT) and threat intel feeds are going berserk, and your team is playing whack-a-mole with potential threats. You’re constantly bombarded with information from countless sources—some reliable, others not so much. Sound familiar? Enter the Admiralty System - a robust framework that can revolutionize how we assess and utilize CTI. But what exactly is this system, and why should you consider implementing it in your CTI program? Let’s dive in.
What the Heck is the Admiralty System Anyway?
The Admiralty System was originally designed for naval intelligence in the early 20th century and has since been adapted by intelligence agencies worldwide. The system was developed to ensure the information and intelligence received by the Admiralty in the British Royal Navy (hence the name ‘Admiralty System') were uniform and standardized. This allowed them to compare the various pieces of information – often received weeks or months apart – about the same observation. (Yes, there was a time without the internet, gasp!) Playfully, we can call this system a sort of ‘BS detector’ for intelligence. Simply put, it's all about figuring out if your intel sources are credible and if the information the ‘other side’ is feeding you is genuine or just hot air and lies. Ludo Block provides additional insights into the Admiralty System via his blog, The Origin of Information Grading Systems.
Breaking it Down: The Admiralty System 101
The Admiralty System, NATO AJP-2.1 is a structured framework used to evaluate the quality of your sources and the information they provide. The framework does this by assessing two distinct components, Source Reliability and Information Credibility. These two parts are separated to ensure a clear and unbiased data assessment, allowing for more informed decision-making.
It uses alphanumeric codes to rate these two crucial aspects:
- Source Reliability, rated from A (Completely reliable) to F (Cannot be judged), refers to the trustworthiness of the origin of the information. By evaluating the source independently of the data itself, analysts can assess whether a particular piece of intelligence is likely to be accurate based on the source’s history, skills, knowledge of the subject, how close they were to the incident, and many other attributes.
- Information Credibility, rated from 1 (Confirmed by other sources) to 6 (Truth cannot be judged), refers to the trustworthiness of the data provided, regardless of the source. This is important! More on that below. The assessment of the information considers whether the information makes sense, is consistent with other known facts, and has been corroborated by other independent sources.
Now, let’s make that easier to understand:
- Source Reliability: This is your trust-o-meter, ranging from A (solid gold) to F (who knows/I haven’t checked yet).
- Information Credibility: This is your truth scale, from 1 (multiple independent sources confirm) to 6 (I haven’t even bothered to check it yet; it could be fact, it could be fiction).
So, when you see something rated as ‘B2’ or above, you know their information and assessments SHOULD be based on rock-solid sources and other folks are backing it up. B2 and above is the kind of intel you want to bet on, but it is harder to get than you would think.
Please note! Source reliability and information credibility must be evaluated separately to avoid bias. Although a source has a history of being trustworthy or untrustworthy, it does not mean the information we are looking at right now is more or less correct! We should make every effort to assess the two separately which allows analysts to focus on the data's accuracy and logical consistency, independent of where it came from. I’ll talk more about this later.
Cybersecurity Makeover: Admiralty System CTI Edition
While the Admiralty System wasn't originally designed for cybersecurity, its principles are remarkably adaptable to the digital realm. For example, the Admiralty System is included as a Malware Information Sharing Platform (MISP) taxonomy called the ‘Admiralty Scale,’ which allows someone to tag the assessed source and information within the MISP open-source threat intelligence platform. Now, let's give this old-school system a cyber facelift.
Source Reliability in CTI
A: Verified technical indicators (e.g., malware hashes from trusted sandboxes). This is your top-shelf stuff that has a history of reliability. Think VirusTotal and Proofpoint are both confirming your malware hash, and you have confirmed they are not using the same datasets.
B: Reputable threat intelligence feeds or intelligence shared in closed communities by researchers you've come to trust over time because they are mostly correct.
C: Open-source intelligence from security researchers, for instance, that security researcher on X/Twitter who is often correct.
D: Unverified dark web chatter from a sketchy user on a forum who’s made past claims that sometimes turned out to be true.
E: Potentially compromised or deceptive sources that have a history of making false claims. Think bots part of a mis- or disinformation campaign.
F: New or unknown sources, such as a new handle/user or tool you're not sure about or that you have not verified yet.
Information Credibility in CTI
- Multiple independent and trusted sources confirm that a ransomware strain is active and has been observed by other independent sources.
- Not yet confirmed, but consistent with other known threat intelligence that lines up with other known intel about the threat actor group you are investigating.
- Not confirmed, somewhat consistent with other intelligence and fits the Modus Operandi of that hacktivist group, but it needs more digging and requires further verification.
- Possible new zero-day that hasn't been confirmed yet. It may be plausible given the current threat landscape and known activities and requires more investigation.
- Improbable based on current understanding. For example, claims that don't match known tactics, techniques, and procedures (TTPs) of the threat actor.
- A post about a massive breach with zero details to back it up, you are unable to verify the claim due to a lack of context or conflicting information.
Why Implement the Admiralty System in Your CTI Program? Why Should You Care?
The Perks of Getting Admiralty-Savvy
The Admiralty System provides you with a framework for assessing the quality of the threat intelligence you read every day. This enables security teams to quickly prioritize threats and allocate resources more effectively. Simply put, cut through the noise! Imagine your threat intelligence flagging a threat, and the source is assessed as ‘A1.’ That's your cue to drop everything and act. No more guessing games.
Wouldn’t it be great if we spoke the same "language" when we shared our source and information assessments? Would that improve your information sharing and collaboration? With the Admiralty System, we have a common language for sharing threat intelligence internally and externally. When your buddy from another incident response team or security operations center (SOC) says they've got a new malware strain and the Admiralty System score is ‘B2,’ you know exactly what they mean (as long as we use the same unaltered framework). Sharing is caring, right?
Work smarter, Not harder! Why chase every rabbit down the hole? Focus on the high-priority stuff and let the ‘E5’ noise fade into the background. Not all threats are created equal. The Admiralty System offers you a way to focus your efforts on the most reliable and credible threats, optimizing your resource allocation.
Automate like a boss! The structured nature of the Admiralty System ratings makes it easier to integrate with automated systems. You can set up rules in your security information and event management (SIEM) system or in your MISP to automatically escalate high-priority threats based on their source and information ratings. Essentially, anything rated ‘B2’ or higher could be escalated. Let the machines handle the grunt work while you focus on the big picture. (PS! This requires a high level of trust.)
Real-World CTI Scenarios: Admiralty System in Action
Example 1: The Reliable Threat Report
Imagine you receive a report from a top-tier cybersecurity firm (let’s call them CyberPro). CyberPro has consistently provided accurate and timely information in the past, so you’d rate the source as an A, reliable. The report claims that a new zero-day vulnerability in widely used software is being actively exploited, and this claim is backed up by multiple other trusted sources. In this case, you’d rate the information as a 1, credible.
Admiralty Rating: A1—completely reliable source, confirmed information.
Action: If this vulnerability is relevant to you, immediate steps are required, like patching the vulnerability or deploying protective measures.
Example 2: The Dubious Social Media Claim
Now, let’s say you see a tweet from an unknown account claiming that a major financial institution has been hacked. The account has no track record and could easily be spreading misinformation. You’d rate the source as E, unreliable. Since no other credible sources have confirmed this claim, you’d rate the information as 4, improbable.
Admiralty Rating: E4—unreliable source, improbable information.
Action: You might monitor the situation but wouldn’t take any immediate action unless further evidence emerges.
Example 3: The New Kid on the Block
You come across chatter on a Dark Web forum about a new ransomware-as-a-service (RaaS) operation. The source is a known Dark Web user who has previously provided accurate information, but their identity and true motivations are unclear—so you’d rate the source as D, not usually reliable. The information itself suggests that this new RaaS operation is gaining traction, but there’s no independent verification or concrete evidence at this stage, so you rate the information as 4, doubtful.
Admiralty Rating: D4—not usually reliable source, doubtful information.
Action: If this is important to you, keep the information on your radar and monitor for further developments or corroboration from more reliable sources, but do not take immediate action unless more credible evidence emerges.
Implementing the Admiralty System: Best Practices
You may be tempted to adapt the Admiralty System to your needs. Don’t! The Admiralty System is universal, and the specific criteria for each rating should not vary depending on your organization or your needs. If you do that, your rating (B2) will not match up with any other rating by other researchers. If everyone did that, we would have to know the varying way each source, vendor and researcher uses the system, which would create a huge mess. You and your team should take the time to define what each rating means in the context of your organization and automate it, but do not change it!
As with everything, start small. Train your team and run a tabletop exercise. Throw some real-world scenarios at them and see how they rate them. Compare those findings and fine-tune your overall understanding of how they should assess a source and the information. Ensure that all team members understand how to use the Admiralty System consistently. It's like calibrating your collective BS detector.
Integrate, integrate, integrate! Look for ways to incorporate the Admiralty System ratings into your existing security tools and workflows. This might involve custom fields or specific tags in your threat intelligence platform. But slap those admiralty ratings onto every piece of intel that comes through your CTI program. Make it as natural as adding tags or categories.
Update, rinse, repeat! This is important and a point that many forget (or don’t want) to do. The cyber threat landscape is constantly changing. Regularly review your ratings and criteria to ensure they remain relevant. That source you rated ‘B’ last month? Maybe they've slipped or even improved since then. Don't be afraid to demote or promote your sources as things change. Remember, this is a dynamic process, and we must reassess our scores as we get new information.
Also, don’t forget that a source you rated as ‘A’ on one topic does not automatically make it an ‘A’ on another topic or different context! You have to assess the source separately each time!
Remember, in the end, the Admiralty System is a tool to aid decision-making. It helps decision-makers and readers of your reporting understand the foundation of your assessment or warning and further explains why they should care.
The Not-So-Rosy Parts: Challenges and Considerations
While the Admiralty System offers numerous benefits, it's not without its challenges in the context of CTI.
Cyber threats evolve quickly, which can make it difficult to maintain accurate source reliability ratings. That new ransomware group? They might hop from ‘F6’ to ‘B2’ in a matter of days. You must stay on your toes and update your source statements and assessments as the threat intelligence changes.
The sheer amount of threat data, information, feeds, and reporting can be overwhelming. You might need to automate initial ratings just to keep up with the fire hose of data and then manually assess the source and information as you use it in your reporting and then provide higher weighting over time. Machine learning, anyone?
Despite the standardized framework, there's still an element of subjectivity in assigning ratings. Bob from the SOC might rate things differently than Alice from CTI. Regular calibration sessions are your friend and can help maintain consistency. However, note that these differing opinions and perspectives are also a good thing! You are likely to have an unbiased and more accurate rating over time when there are differing analysts providing inputs on a particular source or information.
Over-reliance is a deadly sin; unreliable sources can sometimes provide accurate information, and trusted sources can occasionally get it wrong. By evaluating the source and the information separately, you can avoid falling into the trap of automatically dismissing or accepting data based on who or where it comes from. In other words, do not let an ‘A1’ rating lull you into a false sense of security. Always trust, but verify, even when you trust the source completely. The rating should inform, not dictate, your trust in the source and information.
Wrapping It Up: Your Cyber Intel, Leveled Up
Look, in this crazy world of cyber threats, the Admiralty System isn't just nice to have - it's your edge. It's about making sense of the chaos, focusing on what matters, and making decisions with confidence.
The Admiralty System, when adapted for CTI, offers a robust framework for enhancing the reliability and actionability of your intelligence. Sure, it'll take some effort to get it rolling. You might need to tweak it, train your team, and integrate it into your workflow. But trust me, when you're effortlessly prioritizing threats, collaborating like a pro, and making informed decisions at the speed of cyber, you'll wonder how you ever managed without it.
So, ready to give your cyber threat intel game a serious upgrade? The Admiralty System is waiting. Let's do this!
Take your cyber threat intelligence skills to the next level with SANS FOR589: Cybercrime Intelligence. Dive deep into the strategies discussed here, master the Admiralty System, and learn how to apply it in real-world cybercrime investigations. Don’t wait—register today for FOR589 or sign up for a live demo to see how this course can transform your approach to threat intelligence!
