New

SEC497: Practical Open-Source Intelligence (OSINT)

  • In Person (6 days)
  • Online
36 CPEs

SEC497 is based on two decades of experience with open-source intelligence (OSINT) research and investigations supporting law enforcement, intelligence operations, and a variety of private sector businesses ranging from small start-ups to Fortune 100 companies. The goal is to provide practical, real-world tools and techniques to help individuals perform OSINT research safely and effectively. One of the most dynamic aspects of working with professionals from different industries worldwide is getting to see their problems and working with them to help solve those problems. SEC497 draws on lessons learned over the years in OSINT to help others. The course not only covers critical OSINT tools and techniques, it also provides real-world examples of how they have been used to solve a problem or further an investigation. Hands-on labs based on actual scenarios provide students with the opportunity to practice the skills they learn and understand how those skills can help in their research. 29 Hands-on Labs + Capstone CTF

What You Will Learn

SEC497 is a comprehensive training course on Open-Source Intelligence (OSINT) written by an industry professional with over two decades of experience. The course is designed to teach you the most important skills, tools, and methods needed to launch or further refine your investigation skills. SEC497 will provide actionable information to students throughout the OSINT world, including intelligence analysts, law enforcement officials, cyber threat intelligence and cyber defenders, pen testers, investigators, and anyone else who wants to improve their OSINT skills. There is something for everyone, from newcomers to experienced practitioners.

SEC497 focuses on practical techniques that are useful day in and day out. This course is constructed to be accessible for those new to OSINT while providing experienced practitioners with tried-and-true tools that they can add to their arsenal to solve real-world problems. The course has a strong focus on understanding how systems work to facilitate informed decisions, and includes hands-on exercises based on actual scenarios from the government and private sectors. We will discuss cutting-edge research and outlier techniques and not only talk about what is possible, we will practice doing it! Dive into the course syllabus below for a detailed breakdown of the topics covered.

BUSINESS TAKEAWAYS:

This course will help your organization:

  • Improve the effectiveness, efficiency, and success of OSINT investigations
  • Build an OSINT team that can perform a variety of OSINT investigations while practicing good OPSEC
  • Create accurate reporting of your organization's online infrastructure
  • Understand how breach data can be used for offensive and defensive purposes

You will be able to:

  • Perform a variety of OSINT investigations while practicing good OPSEC
  • Create sock puppet accounts
  • Locate information on the internet, including some hard-to-find and deleted information
  • Locate individuals online and examine their online presence
  • Understand and effectively search the dark web
  • Create an accurate report of the online infrastructure for cyber defense, merger and acquisition analysis, pen testing, and other critical areas for an organization.
  • Use methods that can often reveal who owns a website as well as the other websites that they own or operate
  • Understand the different types of breach data available and how they can be used for offensive and defensive purposes
  • Effectively gather and utilize social media data
  • Understand and use facial recognition and facial comparison engines
  • Quickly and easily triage large datasets to learn what they contain
  • Identify malicious documents and documents designed to give away your location

Hands-On Labs

SEC497 has 29 hands-on exercises based on actual scenarios for the government and private sector.

You will receive with this course:

  • A Linux virtual machine complete with electronic workbook

Syllabus (36 CPEs)

  • Overview

    Before diving into tools and techniques to find, gather, and process information, the course starts with a discussion of how to undertake these activities as safely and effectively as possible. This section begins with an overview of the OSINT process and tips on avoiding analytical pitfalls. We then move into Operational Security or OPSEC. This isn't just a list of browser plugins and VPN providers: we'll look at what truly matters when managing attribution. Most of us don't have unlimited budgets and can't achieve "perfect OPSEC" - the good news is that most of us don't have to! We'll discuss the major risks and ways to mitigate them within a reasonable budget.

    A big part of OSINT is going to new sites and downloading files and information. We'll discuss free online resources that can be used to determine if a site is a known security risk before you visit it and to analyze files to determine if they have malicious content. We'll discuss how canary tokens work and create our own with a hands-on lab. Canary tokens are files that can give away our actual IP address and other information about our systems without our knowledge.

    Creating fictitious accounts (aka sock puppets) has gotten tougher over the past few years, with many sites requiring criteria like a real phone number, facial image, etc. We'll discuss the issues and cover current methods for creating these accounts.

    The course section wraps up by examining two tools that can improve your organization and efficiency. Hunchly is a fantastic tool for cataloging online research, and Obsidian is an effective open-source tool for note-taking and various other uses. We'll also cover report writing.

    Many OSINT students have improving Linux skills on their to-do list, so at the end of the section there is an optional lab for Linux command line practice. This gives students who would like to work on these skills the opportunity to do so in a controlled environment.

    Exercises
    • Managing Your Attribution
    • Dealing with Potential Malware
    • Canary Tokens
    • Hunchly
    • Obsidian
    • [Optional] Linux Command Line Practice
    Topics
    • The OSINT Process
    • Avoiding Analytical Pitfalls
    • OPSEC
    • Dealing with Potential Malware
    • Canary Tokens
    • Creating Accounts
    • Hunchly
    • Effective Note Taking
    • Report Writing
    • Introduction to Linux
  • Overview

    Section Two presents a range of fundamental skills that all OSINT practitioners should have, regardless of the industry they work in. We'll start with a brief overview of curated lists of OSINT resources and quickly move into understanding the fundamentals of how the web works and utilizing search engines effectively. We'll cover methods to find other sites owned and operated by the same individuals, how to see content that the site owners may not want you to see, and, as always, the OPSEC implications and how to do undertake these tasks safely. We'll also cover the why and how of setting up persistent monitoring alerts.

    Multiple methods will be presented to archive content from websites, view historical content from websites, and get other sites to visit websites on your behalf. We'll talk about collecting and preserving Internet data and how to convert raw data into useable formats for processing and analysis. Well discuss how to gather useful intelligence from metadata, even if the data initially appear insignificant or do not appear at all, and look at useful sites for mapping, imagery, and analysis.

    The course section will then turn to image analysis, with a discussion of methodology, tools that can help us, and some real-world examples. From there, we'll move into facial recognition and real-world examples and resources we can use to find people online. We'll conclude with a discussion about translation resources.

    At the end of the section there will be an optional capstone. Participants will start off with raw chat logs from a Russian ransomware group and go through the process of converting the logs into a usable format for analysis.

    Exercises
    • Search
    • Instant Data Scraper
    • Metadata
    • Reverse Image Search
    • Facial Recognition
    • Translation
    • [Optional] Day 2 Capstone
    Topics
    • OSINT Link and Bookmark Collections
    • Web Fundamentals and Search Engines
    • Web Archives and Proxy Sites
    • Collecting and Processing Web Data
    • Metadata
    • Mapping
    • Image Analysis and Reverse Image Searches
    • Facial Recognition
    • Translations
  • Overview

    Section Three of the course focuses on investigating individuals or groups. We'll start by discussing privacy and then get into techniques to research usernames and email addresses across popular sites to discover an individuals accounts. The section then covers how to determine if email addresses are potentially tied to fraud and the places where the individual(s) connected to the email addresses may have been.

    Many OSINT investigations start with a selector such as a phone number or address and require that the researcher tie that selector to an individual or group. We'll cover numerous resources and techniques you can use to do this, including some that can help identify the owner of a prepaid phone number.

    The remainder of the section will focus on social media sites, including advanced Facebook searches and ways to see deleted Twitter data and analyze Twitter bots. We'll also cover methods to view content on social media sites when you don't have an account on that site; searching and analyzing alternative social media sites; geolocation of social media data; and trends, sentiment, and reputation.

    Exercises
    • Researching Usernames
    • Keybase
    • Email
    • Twitter
    • Twitter Bot Analysis
    Topics
    • Privacy
    • Usernames
    • Email Addresses
    • Addresses and Phone Numbers
    • Introduction to Social Media
    • Facebook
    • Twitter
    • Other Social Media Sites
    • Geolocation
    • Trends, Sentiment, and Bots
  • Overview

    Section Four covers investigating websites, IP addresses, and other infrastructure, including the cloud. For students who don't consider themselves tech savvy, we'll take the time to explain what the elements are and how they work, and well provide numerous real-world examples of how these elements have helped in investigations. This course section is critical even for analysts who don't focus on technical topics because understanding how these technical elements work reduces the likelihood of falling down rabbit holes during their research.

    For students who focus more on technology topics, such as those who work in Cyber Threat Intelligence, we'll cover a variety of tools and resources to learn as much as we can about such topics as:

    • Where in the world an IP address is located, and whether it is a VPN/proxy/Tor node
    • Why IPv6 is different and why that might be good for your investigation
    • Historical WHOIS data
    • Understanding DNS records
    • Why certificate transparency is excellent for defenders&and attackers
    • What we can learn from email headers
    • Subdomains for enumeration and, potentially, de-anonymization
    • Technology-focused search engines like Shodan and Censys
    • Cyber Threat Intelligence
    • Finding sensitive data in the cloud

    This course section is a mix of understanding how things work, studying real-world examples and case studies, looking at some cutting-edge research, and using tools in creative ways to achieve things most people did not know were possible.

    Exercises
    • IP Address Research
    • WHOIS
    • DNS
    • Amass and Eyewitness
    • Censys and Shodan
    • Buckets of Fun
    Topics
    • IP Addresses
    • Common Ports
    • WHOIS
    • DNS
    • Certificate Transparency
    • Email Headers
    • Subdomains
    • Technology-Focused Search Engines
    • Cyber Threat Intelligence
    • Cloud
  • Overview

    Section Five is a fun mix of topics ranging from researching businesses and transitions to covering wireless for OSINT, including using Wi-Fi names to enrich digital forensics data and research locations. We'll also explore different types of breach data and how it can be used for various OSINT and cyber defender purposes.

    If you work in OSINT long enough, a giant pile of data will eventually be placed in front of you, and someone will ask you whats in it. Depending on your job, this may already be a regular occurrence. This section will cover how to triage and search large datasets effectively and quickly using free or cheap resources.

    We'll also take a deep dive into the dark web, covering how it works, how we can find things, and what we can expect to find. We'll examine a case study of breach data hitting the dark web and tricks we can use to speed up dark web downloads. We'll also have a short section on cryptocurrency that mainly focuses on a resource that allows us to track cryptocurrency transactions with a focus on web 3.0 and NFTs.

    As the course section winds down, we'll talk about different automation options that require no programming. The final portion of the section is called "path forward" and covers a variety of resources that can help you continue your OSINT learning journey.

    Exercises
    • Business
    • Wireless
    • Bulk Data Triage
    • Tor and PGP
    • Breach Data

    Topics
    • Researching Businesses
    • Wireless
    • Breach Data
    • Dealing with Large Datasets
    • Dark Web
    • Cryptocurrency
    • Automation
    • Path Forward
  • Overview

    The capstone for the SEC497 course is a multi-hour capture the flag event which allows students to work together in small groups to create a threat assessment for a fictional client. Preparing this assessment will require that students use the skills learned throughout the course on a variety of real-world sites. The assessment will be delivered to the client (the instructor), who will provide feedback to each group.

Prerequisites

Basic computer knowledge is required for this course.

Laptop Requirements

Important! Bring your own system configured according to these instructions!

A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will likely leave the class unsatisfied because you will not be able to participate in hands-on exercises that are essential to this course. Therefore, we strongly urge you to arrive with a system meeting all the requirements specified for the course.

CRITICAL NOTE: Apple systems using the M1 processor line cannot perform the necessary virtualization functionality and therefore cannot in any way be used for this course.

A properly configured system is required for each student participating in this course. Before coming to class, carefully read and follow these instructions exactly.

Host Operating System: Latest version of Windows 10, macOS 10.15.x or later, or Linux that also can install and run the VMware virtualization products described below.

It is necessary to fully update your host operating system prior to the class to ensure that you have the right drivers and patches installed to utilize the latest USB 3.0 devices.

Those who use a Linux host must also be able to access exFAT partitions using the appropriate kernel or FUSE modules.

You also must have 8 GB of RAM or higher for the VM to function properly in the class.

It is critical that your CPU and operating system support 64-bit so that our 64-bit guest virtual machine will run on your laptop.

In addition to having 64-bit capable hardware, AMD-V, Intel VT-x, or the equivalent must be enabled in BIOS/UEFI.

Download and install either VMware Workstation Pro 15.5.x, VMware Player 15.5.x, or Fusion 11.5.x or higher versions before class. If you do not own a licensed copy of VMware Workstation or Fusion, you can download a free 30-day trial copy from VMware. VMware will send you a time-limited serial number if you register for the trial on its website.

Other virtualization software, such as VirtualBox and Hyper-V, are not appropriate because of compatibility and troubleshooting problems you might encounter during class.

VMware Workstation Pro and VMware Player on Windows 10 are not compatible with Windows 10 Credential Guard and Device Guard technologies. Please disable these capabilities for the duration of the class if they are enabled on your system.

MANDATORY SEC497 SYSTEM REQUIREMENTS:

  • CPU: 64-bit 2.0+ GHz processor or higher-based system is mandatory for this course (Important - Please Read: a 64-bit system processor is mandatory)
  • BIOS/UEFI: VT-x, AMD-V, or the equivalent must be enabled in the BIOS/UEFI
  • RAM: 8 GB (gigabytes) of RAM or higher is mandatory for this course (Important - Please Read: 8 GB of RAM or higher is mandatory)
  • Wireless Ethernet 802.11 G/N/AC
  • USB 3.0 port (courseware provided via USB)
  • Disk: 30 gigabytes of free disk space
  • VMware Workstation Pro 15.5.X+, VMware Player 15.5.X+ or Fusion 11.5+
  • Privileged access to the host operating system with the ability to disable security tools
  • A Linux virtual machine will be provided in class

Your course media will be delivered via download. The media files for class can be large, some in the 10-15 GB range. You need to allow plenty of time for the download to complete. Internet connections and speed vary greatly and depend on many different factors. Therefore, it is not possible to give an estimate of the length of time it will take to download your materials. Please start your course media downloads as soon as you get the link. You will need your course media immediately on the first day of class. Waiting until the night before the class starts to begin your download has a high probability of failure.

If you have additional questions about the laptop specifications, please contact laptop_prep@sans.org.

Author Statement

"When I started the first open-source intelligence (OSINT) unit for my organization over a decade ago, I was told we had no budget for tools, equipment, or training. I used to joke that one nice thing about not having a budget was that it made many of my decisions very easy. If there was something I needed, I either built it myself or did without.

Coming from that background forces you to understand how things work and what truly matters. In addition to performing countless OSINT investigations, I've traveled across the world for over a decade teaching operational security (OPSEC) and OSINT to various government agencies and consulted with numerous private companies, ranging from small start-ups to Fortune 100 enterprises. I have helped hunt down international fugitives, identified online infrastructure for a merger and acquisitions due diligence report, and handled numerous tasks in between. This course allows me me share my experience with what works, what does not work, and how we can achieve our goals with minimal effort and cost."

- Matt Edmondson

Reviews

Very Informative course and provided pointers to numerous breach data sites which could aid numerous investigations.
Kanika Mittal
Very relevant information is provided that can be deployed immediately even by novice users. Excellent!
Shay Christensen
The module on dealing with large data sets was very helpful. Getting a deep understanding on the challenges large data sets pose and how to work around them is very helpful and practical.
Jamal Gumbs
Business intelligence is a topic near and dear to me and Matt did a fantastic job covering not just the how-tos of collecting and analyzing company data, but also providing the real world context.
Sammy Shin
I appreciate the realism in all of these labs. Students can easily turn around and do real world OSINT investigations with many of these labs.
Erich Nieskes

    Register for SEC497

    Loading...