One Week Only! Get an iPad Air with Smart Keyboard, Surface Go, or $300 Off with OnDemand or vLive Training!

SEC487: Open-Source Intelligence (OSINT) Gathering and Analysis

Lots of real world tools that will help improve my job.

Sean McCormack, Bridgewater Associates

Great intro to OSINT

Jason Adamson, CrowdStrike

This is a foundational course in open-source intelligence (OSINT) gathering and, as such, will move quickly through many areas of the field. While the course is an entry point for people wanting to learn about OSINT, the concepts and tools taught are far from basic. The goal is to provide the OSINT groundwork knowledge for students to be successful in their fields, whether they are cyber defenders, threat intelligence analysts, private investigators, insurance claims investigators, intelligence analysts, law enforcement personnel, or just someone curious about OSINT.

Many people think using their favorite Internet search engine is enough to find the data they need and do not realize that most of the Internet is not indexed by search engines. SEC487 teaches students effective methods of finding these data. You will learn real-world skills and techniques that law enforcement, private investigators, cyber attackers, and defenders use to scour the massive amounts of information found on the Internet. Once you have the information, we'll show you how to ensure that it is corroborated, how to analyze what you've gathered, and how to make sure it is useful in your investigations.

You will learn OSINT by completing more than 20 hands-on exercises using the live Internet and dark web.

Notice:

SEC487 students will receive licensing information in the SANS portal account that is linked to their registration. Please ensure that you can access the SANS portal account that is linked to your registration at the start of your course.

If you are registering another individual on behalf of your organization, you must register that individual using the email address that is linked to his or her SANS portal account. That will ensure that the individual can receive licensing information in his or her SANS portal account in order to be prepared with the proper equipment to complete the course (SEC487).

Course Syllabus

Overview

We begin with the basics and answer the questions "what is OSINT" and "how do people use it." This first section of this course is about level-setting and ensuring that all students understand the background behind what we do in the OSINT field. We also establish the foundation for the rest of the course by learning how to document findings and set up an OSINT platform. This information taught in this section is a key component for the success of an OSINT analyst because without these concepts and processes in place, researchers can get themselves into serious trouble during assessments by inadvertently alerting their targets or improperly collecting data.

CPE/CMU Credits: 6

Topics
  • Course Introduction
  • Understanding OSINT
  • Goals of OSINT Collection
  • Diving into Collecting
  • Taking Excellent Notes
  • Determining Your Threat Profile
  • Setting up an OSINT Platform
  • Effective Habits and Process
  • Leveraging Search Engines
Overview

OSINT data collection begins in section two after we get a glimpse of some of the fallacies that could influence our conclusions and recommendations. From this point in the course forward, we examine distinct categories of data and think about what it could mean for our investigations. Retrieving data from the Internet could mean using a web browser to view a page or, as we learn in this section, using command line tools, scripts, and helper applications.

CPE/CMU Credits: 6

Topics
  • Data Analysis Challenges
  • Harvesting Web Data
  • File Metadata Analysis
  • OSINT Frameworks
  • Basic Data: Addresses and Phone Numbers
  • Basic Data: Email Addresses
  • User Names
  • Avatars and Reverse Image Searches
  • Additional Public Data
  • Creating Sock Puppets
Overview

Section three kicks off by examining free and paid choices in people search engines and understanding how to use the data we receive from them. Some of these engines provide social media content in their results. This makes a terrific transition for us to move into social media data, geolocation, and eventually mapping and imagery.

CPE/CMU Credits: 6

Topics
  • People Search Engines
  • Exercise People Searching
  • Facebook Analysis
  • LinkedIn Data
  • Instagram
  • Twitter Data
  • Geolocation
  • Imagery and Maps
Overview

Section four focuses on many different but related OSINT issues. This is our blue team day, as we dive into OSINT for IP addresses, domain names, DNS, and Whois. We then move into how to use wireless network information for OSINT. We end the section with two huge modules on searching international government websites for OSINT data and supporting business processes with OSINT.

CPE/CMU Credits: 6

Topics
  • Whois
  • IP Addresses
  • DNS
  • Finding Online Devices
  • Wireless Networks
  • Recon Tool Suites and Frameworks
  • Government Data
  • Researching Companies
Overview

The beginning of section five focuses on understanding and using three of the dark web networks. Students will learn why people use Freenet, I2P, and Tor. Each network is discussed at length so that students don't just know how and why to use it, but also gain an understanding of how those networks work. With the Tor network being such a big player in the dark web, the course spends extra time diving into its resources.

After tackling the dark web, we examine how we can use breach data in our cases and to address international OSINT issues. We end the section by examining how to find and track vehicles of all sizes.

The end of this section is a massive lab, the Solo Capture-the-Flag (CTF) Challenge that helps students put together all that they have learned up until now in the course. Through a semi-guided walk-through that touches on many of the concepts taught throughout the course, students complete a full OSINT assessment at their own speed. Setting aside time to work through our OSINT process in an organized manner reinforces key concepts and allows students to practice executing OSINT process, procedures, and techniques.

CPE/CMU Credits: 6

Topics
  • The Surface, Deep, and Dark Webs
  • The Dark Web
  • Freenet
  • I2P - Invisible Internet Project
  • Tor
  • Monitoring and Alerting
  • International Issues
  • Vehicle Searches
  • Solo CTF Challenge
Overview

The capstone for the course is a group event that brings together everything that students have learned throughout the course. This is not a "canned" Capture-the-Flag event where specific flags are planted and your team must find them. It is a competition where each team will collect specific OSINT data about certain targets. The output from this work will be turned in as a "deliverable" to the "client" (the instructor and fellow classmates). This multi-hour, hands-on event will reinforce what the students practiced in the Solo CTF in the previous section before and add the complexity of performing OSINT assessments under pressure and in a group.

CPE/CMU Credits: 6

Topics
  • Capstone Capture-the-Flag Event

Additional Information

!! IMPORTANT - BRING YOUR OWN LAPTOP CONFIGURED USING THESE DIRECTIONS!!

A properly configured system is required for each student participating in this course. Before coming to class, carefully read and follow these instructions exactly.

You can use any 64-bit version of Windows, MacOS, or Linux as your core operating system that also can install and run VMware virtualization products. You also must have 8 GB of RAM or higher for the VM to function properly in the class.

It is critical that your CPU and operating system support 64-bit so that our 64-bit guest virtual machine will run on your laptop.

In addition to having 64-bit capable hardware, AMD-V, Intel VT-x, or the equivalent must be enabled in BIOS/UEFI.

Please download and install VMware Workstation 14, VMware Fusion 10, or VMware Workstation Player 14 or higher versions on your system prior to the start of the class. If you do not own a licensed copy of VMware Workstation or Fusion, you can download a free 30-day trial copy from VMware. VMware will send you a time-limited serial number if you register for the trial on its website.

MANDATORY SEC487 SYSTEM REQUIREMENTS:

  • CPU: 64-bit 2.0+ GHz processor or higher-based system is mandatory for this course (Important - Please Read: a 64-bit system processor is mandatory)
  • BIOS/UEFI: VT-x, AMD-V, or the equivalent must be enabled in the BIOS/UEFI
  • RAM: 8 GB (gigabytes) of RAM or higher is mandatory for this course (Important - Please Read: 8 GB of RAM or higher is mandatory)
  • Wireless Ethernet 802.11 G/N/AC
  • USB 3.0 port (courseware provided via USB)
  • Disk: 30 gigabytes of free disk space
  • VMware Workstation 14, Workstation Player 14, or Fusion 10 (or newer)
  • Privileged access to the host operating system with the ability to disable security tools
  • A Linux virtual machine will be provided in class

If you have additional questions about the laptop specifications, please contact laptop_prep@sans.org.

This course will teach you techniques to help your work whether you are trying to find suspects for a legal investigation, identify candidates to fill a job position, gather hosts for a penetration test, or search for honey tokens as a defender.

While this list is far from complete, the OSINT topics in SEC487 will be helpful to:

  • Cyber Incident Responders
  • Digital Forensics (DFIR) analysts
  • Penetration Testers
  • Social Engineers
  • Law Enforcement
  • Intelligence Personnel
  • Recruiters/Sources
  • Private Investigators
  • Insurance Investigators
  • Human Resources Personnel
  • Researchers

SEC487 is a learn it-do it course where we examine a topic and then dive into a hands-on lab to reinforce the learning. The course has more than 20 labs spaced across the first five sections, followed by the final hands-on Capture-the-Flag challenge in section six. Check out the lab content below to get a feel for what you will be doing within our course virtual machines.

Section 1

  • Set up the course virtual machine and configure the VPN that is used to secure all web traffic
  • Use a MindMap tool to document OSINT data and then analyze relationships between people using a data visualization application
  • Set up a password manager to securely store all the passwords that we will need for our sock puppets and other accounts
  • Create a sock puppet account with realistic user-attributes, which will be key to succeeding in some of the other labs later in the course
  • Join a class Slack group to discuss OSINT and the class by way of a lab that walks you through the setup and use of the application

Section 2

  • Harvest web data such as Google Analytics IDs and the information within HTTPS certificates
  • Trace a home address and phone number to their owners
  • Gather email addresses for a company
  • Use a reconnaissance framework to rapidly scan websites looking for specific user accounts
  • Search reverse images to find the identity of the person and other places where that image was used

Section 3

  • Execute queries on search engines to find information about someone
  • Conduct Facebook queries to retrieve surface and deep data
  • Analyze tweets to determine sentiment and discover where the tweets are geolocated
  • Scrape metadata and map GPS coordinates

Section 4

  • Use online mapping sites to recon an area
  • Search for wireless network data and use it to verify an alibi
  • Run an OSINT framework to discover what information can be found about a domain
  • Examine various government websites to answer trivia questions
  • Gather data points about the CEO and the systems used at a business

Section 5

  • Dive into the deep web by using Tor to visit Internet sites and hidden services, and set up our own hidden service
  • Query the HaveIBeenPwned.com website and API to find compromised user accounts
  • Use translation sites to practice translating text into other languages
  • Discover the popular websites and mobile apps used in several countries
  • Undertake the Solo CTF that brings together many of the previous labs and helps students practice process

Section 6

  • Participate in the group Capture-the-Flag competition

  • A USB storage device with a custom Linux virtual machine where all labs will be run from
  • A digital wiki (inside the virtual machine) containing electronic versions of the labs
  • Create an OSINT process
  • Conduct OSINT investigations in support of a wide range of customers
  • Understand the data collection life cycle
  • Create a secure platform for data collection
  • Analyze customer collection requirements
  • Capture and record data
  • Create sock puppet accounts
  • Create your own OSINT process
  • Harvest web data
  • Perform searches for people
  • Access social media data
  • Assess a remote location using online cameras and maps
  • Examine geolocated social media
  • Research businesses
  • Use government-provided data
  • Collect data from the dark web
  • Leverage international sites and tools

Author Statement

"I have always been intrigued by the types and amount of data that are available on the Internet. From researching the best restaurants in a foreign town to watching people via video cameras, it all fascinates me. As the Internet evolved, more high-quality, real-time resources became available and every day was like a holiday, with new and wondrous tools and sites coming online and freely accessible.

"At a certain point, I was no longer in awe of the great resources on the web and, instead, transitioned to being surprised that people would post images of themselves in illegal or compromising positions or that a user profile contained such explicit, detailed content. My wonder shifted to concern for these people. What I found was that, if you looked in the right places, you could find almost anything about a person, a network, or a company. Piecing together seemingly random pieces of data into meaningful stories became my passion and, ultimately, the reason for this course.

"I recognized that the barrier to performing excellent OSINT was not that there was no free data on the Internet. It was that there was too much data on the Internet. The challenge transitioned from 'how do I find something' to 'how do I find only what I need.' This course was born from this need to help others learn the tools and techniques to effectively gather and analyze OSINT data from the Internet."

- Micah Hoffman

Additional Resources

Take your learning beyond the classroom. Explore our site network for additional resources related to this course's subject matter.

Online options available. Train from any location.

21 Training Results
Type Topic Course / Location / Instructor Date Register

Training Event
Security
Dec 12, 2019 -
Dec 17, 2019
 

Summit
Security
Jan 22, 2020 -
Jan 27, 2020
 

Training Event
Security
SANS Las Vegas 2020
Las Vegas, NV
Jan 27, 2020 -
Feb 1, 2020
 

Summit
Security
Feb 19, 2020 -
Feb 24, 2020
 

Training Event
Security
SANS Jacksonville 2020
Jacksonville, FL
Feb 24, 2020 -
Feb 29, 2020
 

Summit
Security
Mar 4, 2020 -
Mar 9, 2020
 

Training Event
Security
SANS St. Louis 2020
St. Louis, MO
Mar 8, 2020 -
Mar 13, 2020
 

Training Event
Security
SANS 2020
Orlando, FL
Apr 5, 2020 -
Apr 10, 2020
 

Training Event
Security
Apr 14, 2020 -
Apr 19, 2020
 

Training Event
Security
Apr 27, 2020 -
May 2, 2020
 

Training Event
Security
May 8, 2020 -
May 13, 2020
 

Training Event
Security
SANS Amsterdam May 2020
Amsterdam, Netherlands
May 11, 2020 -
May 16, 2020
 

Training Event
Security
May 26, 2020 -
May 31, 2020
 

Training Event
Security
Jun 8, 2020 -
Jun 13, 2020
 

Training Event
Security
SANSFIRE 2020
Washington, DC
Jun 15, 2020 -
Jun 20, 2020
 

Training Event
Security
Staff
Jul 20, 2020 -
Jul 25, 2020
 

Training Event
Security
Staff
Aug 3, 2020 -
Aug 8, 2020
 

SelfStudy
Security
Online
Anytime  

OnDemand
Security
Online
Anytime  

Simulcast
Security
Online
Feb 19, 2020 -
Feb 24, 2020
 

Simulcast
Security
Online
Staff
May 26, 2020 -
May 31, 2020
 

*Course contents may vary depending upon location, see specific event description for details.