What You Will Learn
This is a foundational course in open-source intelligence (OSINT) gathering and, as such, will move quickly through many areas of the field. While the course is an entry point for people wanting to learn about OSINT, the concepts and tools taught are far from basic. The goal is to provide the OSINT groundwork knowledge for students to be successful in their fields, whether they are cyber defenders, threat intelligence analysts, private investigators, insurance claims investigators, intelligence analysts, law enforcement personnel, or just someone curious about OSINT.
Many people think using their favorite Internet search engine is enough to find the data they need and do not realize that most of the Internet is not indexed by search engines. SEC487 teaches students effective methods of finding these data. You will learn real-world skills and techniques that law enforcement, private investigators, cyber attackers, and defenders use to scour the massive amounts of information found on the Internet. Once you have the information, we'll show you how to ensure that it is corroborated, how to analyze what you've gathered, and how to make sure it is useful in your investigations.
You will learn OSINT by completing more than 20 hands-on exercises using the live Internet and dark web.
You Will Be Able To
- Create an OSINT process
- Conduct OSINT investigations in support of a wide range of customers
- Understand the data collection life cycle
- Create a secure platform for data collection
- Analyze customer collection requirements
- Capture and record data
- Create sock puppet accounts
- Create your own OSINT process
- Harvest web data
- Perform searches for people
- Access social media data
- Assess a remote location using online cameras and maps
- Examine geolocated social media
- Research businesses
- Use government-provided data
- Collect data from the dark web
- Leverage international sites and tools
SEC487 is a learn it-do it course where we examine a topic and then dive into a hands-on lab to reinforce the learning. The course has more than 20 labs spaced across the first five sections, followed by the final hands-on Capture-the-Flag challenge in section six. Check out the lab content below to get a feel for what you will be doing within our course virtual machines.
- Set up the course virtual machine and configure the VPN that is used to secure all web traffic
- Use a MindMap tool to document OSINT data and then analyze relationships between people using a data visualization application
- Set up a password manager to securely store all the passwords that we will need for our sock puppets and other accounts
- Create a sock puppet account with realistic user-attributes, which will be key to succeeding in some of the other labs later in the course
- Join a class Slack group to discuss OSINT and the class by way of a lab that walks you through the setup and use of the application
- Harvest web data such as Google Analytics IDs and the information within HTTPS certificates
- Trace a home address and phone number to their owners
- Gather email addresses for a company
- Use a reconnaissance framework to rapidly scan websites looking for specific user accounts
- Search reverse images to find the identity of the person and other places where that image was used
- Execute queries on search engines to find information about someone
- Conduct Facebook queries to retrieve surface and deep data
- Analyze tweets to determine sentiment and discover where the tweets are geolocated
- Scrape metadata and map GPS coordinates
- Use online mapping sites to recon an area
- Search for wireless network data and use it to verify an alibi
- Run an OSINT framework to discover what information can be found about a domain
- Examine various government websites to answer trivia questions
- Gather data points about the CEO and the systems used at a business
- Dive into the deep web by using Tor to visit Internet sites and hidden services, and set up our own hidden service
- Query the HaveIBeenPwned.com website and API to find compromised user accounts
- Use translation sites to practice translating text into other languages
- Discover the popular websites and mobile apps used in several countries
- Undertake the Solo CTF that brings together many of the previous labs and helps students practice process
- Participate in the group Capture-the-Flag competition
What You Will Receive
- A Digital Download Package with a custom Linux virtual machine where all labs will be run from
- A digital wiki (inside the virtual machine) containing electronic versions of the labs
Syllabus (36 CPEs)Download PDF
We begin with the basics and answer the questions "what is OSINT" and "how do people use it." This first section of this course is about level-setting and ensuring that all students understand the background behind what we do in the OSINT field. We also establish the foundation for the rest of the course by learning how to document findings and set up an OSINT platform. This information taught in this section is a key component for the success of an OSINT analyst because without these concepts and processes in place, researchers can get themselves into serious trouble during assessments by inadvertently alerting their targets or improperly collecting data.
- Course Introduction
- Understanding OSINT
- Goals of OSINT Collection
- Diving into Collecting
- Taking Excellent Notes
- Determining Your Threat Profile
- Setting up an OSINT Platform
- Effective Habits and Process
- Leveraging Search Engines
OSINT data collection begins in section two after we get a glimpse of some of the fallacies that could influence our conclusions and recommendations. From this point in the course forward, we examine distinct categories of data and think about what it could mean for our investigations. Retrieving data from the Internet could mean using a web browser to view a page or, as we learn in this section, using command line tools, scripts, and helper applications.
- Data Analysis Challenges
- Harvesting Web Data
- File Metadata Analysis
- OSINT Frameworks
- Basic Data: Addresses and Phone Numbers
- Basic Data: Email Addresses
- User Names
- Avatars and Reverse Image Searches
- Additional Public Data
- Creating Sock Puppets
Section three kicks off by examining free and paid choices in people search engines and understanding how to use the data we receive from them. Some of these engines provide social media content in their results. This makes a terrific transition for us to move into social media data, geolocation, and eventually mapping and imagery.
- People Search Engines
- Exercise People Searching
- Facebook Analysis
- LinkedIn Data
- Twitter Data
- Imagery and Maps
Section four focuses on many different but related OSINT issues. This is our blue team day, as we dive into OSINT for IP addresses, domain names, DNS, and Whois. We then move into how to use wireless network information for OSINT. We end the section with two huge modules on searching international government websites for OSINT data and supporting business processes with OSINT.
- IP Addresses
- Finding Online Devices
- Wireless Networks
- Recon Tool Suites and Frameworks
- Government Data
- Researching Companies
The beginning of section five focuses on understanding and using three of the dark web networks. Students will learn why people use Freenet, I2P, and Tor. Each network is discussed at length so that students don't just know how and why to use it, but also gain an understanding of how those networks work. With the Tor network being such a big player in the dark web, the course spends extra time diving into its resources.
After tackling the dark web, we examine how we can use breach data in our cases and to address international OSINT issues. We end the section by examining how to find and track vehicles of all sizes.
The end of this section is a massive lab, the Solo Capture-the-Flag (CTF) Challenge that helps students put together all that they have learned up until now in the course. Through a semi-guided walk-through that touches on many of the concepts taught throughout the course, students complete a full OSINT assessment at their own speed. Setting aside time to work through our OSINT process in an organized manner reinforces key concepts and allows students to practice executing OSINT process, procedures, and techniques.
- The Surface, Deep, and Dark Webs
- The Dark Web
- I2P - Invisible Internet Project
- Monitoring and Alerting
- International Issues
- Vehicle Searches
- Solo CTF Challenge
The capstone for the course is a group event that brings together everything that students have learned throughout the course. This is not a "canned" Capture-the-Flag event where specific flags are planted and your team must find them. It is a competition where each team will collect specific OSINT data about certain targets. The output from this work will be turned in as a "deliverable" to the "client" (the instructor and fellow classmates). This multi-hour, hands-on event will reinforce what the students practiced in the Solo CTF in the previous section before and add the complexity of performing OSINT assessments under pressure and in a group.
- Capstone Capture-the-Flag Event
GIAC Open Source Intelligence
“As the first and only non-vendor specific, industry-wide OSINT certification, the GIAC Open Source Intelligence (GOSI) certification represents a huge milestone in the worlds of open source intelligence and cyber reconnaissance. It creates a marker from which students can be recognized for their achievements and competence in the OSINT field of study. Whether they are performing social media analysis of a target or just “fancy googling,” the GOSI certification shows they have a strong foundation in OSINT.” - Micah Hoffman, SEC487 Course Author
Open Source Intelligence Methodologies and Frameworks
OSINT Data Collection, Analysis, and Reporting
Harvesting Data from the Dark Web
Important! Bring your own system configured according to these instructions!
We ask that you do 5 things to prepare prior to class start. This early preparation will allow you to get the most out of your training. One of those five steps is ensuring that you bring a properly configured system to class. This document details the required system hardware and software configuration for your class. You can also watch a series of short videos on these topics at the following web link https://sansurl.com/sans-setup-videos.
A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will likely leave the class unsatisfied because you will not be able to participate in hands-on exercises that are essential to this course. Therefore, we strongly urge you to arrive with a system meeting all the requirements specified for the course.
A properly configured system is required for each student participating in this course. Before coming to class, carefully read and follow these instructions exactly.Host Operating System: Latest version of Windows 10, macOS 10.15.x or later, or Linux that also can install and run VMware virtualization products described below.
It is necessary to fully update your host operating system prior to the class to ensure you have the right drivers and patches installed to utilize the latest USB 3.0 devices.
Those who use a Linux host must also be able to access exFAT partitions using the appropriate kernel or FUSE modules.
You also must have 8 GB of RAM or higher for the VM to function properly in the class.
It is critical that your CPU and operating system support 64-bit so that our 64-bit guest virtual machine will run on your laptop.
In addition to having 64-bit capable hardware, AMD-V, Intel VT-x, or the equivalent must be enabled in BIOS/UEFI.
Download and install either VMware Workstation Pro 15.5.x, VMware Player 15.5.x or Fusion 11.5.x or higher versions before class. If you do not own a licensed copy of VMware Workstation or Fusion, you can download a free 30-day trial copy from VMware. VMware will send you a time-limited serial number if you register for the trial at their website.
Other virtualization software, such as VirtualBox and Hyper-V, are not appropriate because of compatibility and troubleshooting problems you might encounter during class.
VMware Workstation Pro and VMware Player on Windows 10 is not compatible with Windows 10 Credential Guard and Device Guard technologies. Please disable these capabilities for the duration of the class, if they're enabled on your system, by following instructions in this document.
MANDATORY SEC487 SYSTEM REQUIREMENTS:
- CPU: 64-bit 2.0+ GHz processor or higher-based system is mandatory for this course (Important - Please Read: a 64-bit system processor is mandatory)
- BIOS/UEFI: VT-x, AMD-V, or the equivalent must be enabled in the BIOS/UEFI
- RAM: 8 GB (gigabytes) of RAM or higher is mandatory for this course (Important - Please Read: 8 GB of RAM or higher is mandatory)
- Wireless Ethernet 802.11 G/N/AC
- USB 3.0 port (courseware provided via USB)
- Disk: 30 gigabytes of free disk space
- VMware Workstation Pro 15.5.X+, VMware Player 15.5.X+ or Fusion 11.5+
- Privileged access to the host operating system with the ability to disable security tools
- A Linux virtual machine will be provided in class
Your course media will now be delivered via download. The media files for class can be large, some in the 40 - 50 GB range. You need to allow plenty of time for the download to complete. Internet connections and speed vary greatly and are dependent on many different factors. Therefore, it is not possible to give an estimate of the length of time it will take to download your materials. Please start your course media downloads as you get the link. You will need your course media immediately on the first day of class. Waiting until the night before the class starts to begin your download has a high probability of failure.
SANS has begun providing printed materials in PDF form. Additionally, certain classes are using an electronic workbook in addition to the PDFs. The number of classes using eWorkbooks will grow quickly. In this new environment, we have found that a second monitor and/or a tablet device can be useful by keeping the class materials visible while the instructor is presenting or while you are working on lab exercises.
"I have always been intrigued by the types and amount of data that are available on the Internet. From researching the best restaurants in a foreign town to watching people via video cameras, it all fascinates me. As the Internet evolved, more high-quality, real-time resources became available and every day was like a holiday, with new and wondrous tools and sites coming online and freely accessible.
"At a certain point, I was no longer in awe of the great resources on the web and, instead, transitioned to being surprised that people would post images of themselves in illegal or compromising positions or that a user profile contained such explicit, detailed content. My wonder shifted to concern for these people. What I found was that, if you looked in the right places, you could find almost anything about a person, a network, or a company. Piecing together seemingly random pieces of data into meaningful stories became my passion and, ultimately, the reason for this course.
"I recognized that the barrier to performing excellent OSINT was not that there was no free data on the Internet. It was that there was too much data on the Internet. The challenge transitioned from 'how do I find something' to 'how do I find only what I need.' This course was born from this need to help others learn the tools and techniques to effectively gather and analyze OSINT data from the Internet."
- Micah Hoffman
"Micah Hoffman is the best SANS instructor I have worked with to date. He is a genuine subject matter expert and an outstanding speaker. He possesses a rare ability to be technically proficient and able to explain complicated processes and techniques in terms that anyone can understand." - David U., US Military