FOR572: Advanced Network Forensics: Threat Hunting, Analysis, and Incident Response™

Overview

Although many fundamental network forensic concepts align with those of any other digital forensic investigation, the network presents many nuances that require special attention. Today you will learn how to apply what you already know about digital forensics and incident response to network-based evidence. You will also become acclimated to the basic tools of the trade.

Network data can be preserved, but only if captured or documented while in transit. Whether tactical or strategic, packet capture methods are quite straightforward. While we will use packet capture evidence to explore useful network artifacts in their original form, long-term full-packet capture is still uncommon in most environments. Therefore, many artifacts that can tell us about what happened on the wire in the past come from devices that manage network functions. You will learn about what kinds of devices can provide valuable evidence and at what level of granularity. We will walk through collecting evidence from one of the most common sources of network evidence - a web proxy server - then you'll go hands-on to find and extract stolen data from the proxy.

Exercises

• tcpdump and Wireshark Hands-On (Bonus lab)
• Proxy Log and Cache Analysis (Walk-through)
• Carve Exfiltrated Data

Topics

• Evaluating Web Proxy Data
  • Role of a web proxy
  • Proxy solutions - commercial and open-source
  • Squid proxy server
  • Configuration
    • Logging
    • Automated analysis
    • Cache extraction
• Network Evidence Acquisition
  • Three core types: full-packet capture, Logs, NetFlow
  • Capture devices: switches, taps, Network Packet Brokers, Layer 7 sources, NetFlow
  • Planning to capture: strategies; commercial and home-built platforms
• Network Challenges and Opportunities
  • Challenges provided by a network environment
  • Future trends that will affect network forensics
• Hypertext Transfer Protocol (HTTP) Part 1: Protocol
  • Forensic value
  • Request/response dissection
  • Useful HTTP fields
  • HTTP tracking cookies
  • HTTP/2 and HTTP/3 differences
  • Artifact extraction

Overview

There are countless network protocols that may be in use in a production network environment. We will cover those that are most likely to benefit the forensicator in typical casework, as well as several that help demonstrate analysis methods useful when facing new, undocumented, or proprietary protocols. By learning the "typical" behaviors of these protocols, we can more readily identify anomalies that may suggest misuse of the protocol for nefarious purposes. These protocol artifacts and anomalies can be profiled through direct traffic analysis as well as through the log evidence created by systems that have control or visibility of that traffic. While this affords the investigator with vast opportunities to analyze the network traffic, efficient analysis of large quantities of source data generally requires tools and methods designed to scale.

Knowing how protocols appear in their normal use is critical if investigators are to identify anomalous behaviors. By looking at some of the more frequently used and high-impact network communication protocols, we will specifically focus on the ways in which they can be easily misused by an adversary or a malware author.

While no single course could exhaustively cover the dizzying list of protocols used in a typical network environment, you will build the skills needed to learn whatever new protocols may come your way. The ability to "learn how to learn" is critical, as new protocols are developed every day. Advanced adversaries also develop their own protocols. As you will see later in this class, successfully understanding and counteracting an adversary's undocumented protocol is a similar process to learning those you will see in this section.

Log data is one of the unsung heroes in the realm of network forensics. While the near-perfect knowledge that comes with full-packet capture seems ideal, it suffers from several shortfalls. It is often unavailable, as many organizations have not yet deployed or cannot deploy comprehensive collection systems. When they are in use, network capture systems quickly amass a huge volume of data, which is often difficult to process effectively and must be maintained in a rolling buffer covering just a few days or weeks. The increasing use of encryption for most network traffic also provides a significant barrier to analysis when using full-packet capture, leaving logs from a terminal point of the communication as the artifact with the most potential impact.

In this section, you will learn various logging mechanisms available to both endpoint and network transport devices. You will also learn how to consolidate log data from multiple sources, providing a broad corpus of evidence in one location. As the volume of log data increases, so does the need to consider automated analytic tools. You'll use the SOF-ELK® platform for post-incident log aggregation and analysis, bringing quick and decisive insight to a compromise investigation.

Exercises

• HTTP Profiling
• DNS Profiling, Anomalies, and Scoping
• SOF-ELK Log Aggregation and Analysis

Topics

• Hypertext Transfer Protocol (HTTP) Part 2: Logs
  • Log formats
  • Expanded mod_forensic logging
  • Analysis methods
• Domain Name Service (DNS): Protocol and Logs
  • Architecture and core functionality
  • Fast flux and domain name generation algorithms (DGAs)
  • Logging methods and formats
  • DNS evolution and adaptations
• Forensic Network Security Monitoring (NSM)
  • NSM's emergence from Intrusion Detection Systems (IDSs)
  • Zeek NSM platform
    • Proactive/live use case
    • Post-incident DFIR use case
    • Logs created and formats used
  • JSON parsing with the "jq" utility
  • Community-ID flow hash value
• Logging Protocols and Aggregation
  • Syslog
    • Dual role: server and protocol
    • Source and collection platforms
    • Event dissection

    • rsyslog configuration

  • Microsoft Windows Event Forwarding
    • Deployment model and capabilities
    • Windows Event Forwarding
    • Architecture
    • Analysis mode
  • Log Data Collection, Aggregation, and Analysis
    • Benefits of aggregation: scale, scope, independent validation, efficiency
    • Known weaknesses and mitigations
    • Evaluating a comprehensive log aggregation platform
• Elastic Stack and the SOF-ELK® Platform
  • Basics and pros/cons of the Elastic stack
  • SOF-ELK®
    • Inputs
    • Log-centric dashboards
    • Use as a data exploration platform

Overview

Network connection logging, commonly called NetFlow, may be the single most valuable source of evidence in network investigations. Many organizations have extensive archives of flow data due to its minimal storage requirements. Since NetFlow does not capture any content of the transmission, many legal issues with long-term retention are mitigated. Even without content, NetFlow provides an excellent means of guiding an investigation and characterizing an adversary's activities from pre-attack through operations. Whether for moving within a victim's environment or for data exfiltration, adversaries must move their quarry around through the use of various file access protocols. By knowing some of the more common file access and transfer protocols, a forensicator can quickly identify an attacker's theft actions.

Just as even a fuzzy photo can provide valuable leads in a traditional investigation, NetFlow data can provide a network forensicator with extremely high-value intelligence about network communications. The key to extracting that value is in knowing how to use NetFlow evidence to drive more detailed investigative activities.

NetFlow is also an ideal technology to use in baselining typical behavior of an environment, and therefore, deviations from that baseline that may suggest malicious actions. Threat hunting teams can also use NetFlow to identify prior connections consistent with newly-identified suspicious endpoints or traffic patterns.

In this section, you will learn the contents of typical NetFlow protocols, as well as common collection architectures and analysis methods. You'll also learn how to distill full-packet collections to NetFlow records for quick initial analysis before diving into more cumbersome pcap files.

You'll then examine the File Transfer Protocol, including how to reconstruct specific files from an FTP session. While FTP is commonly used for data exfiltration, it is also an opportunity to refine protocol analysis techniques, due to its multiple-stream nature.

Lastly, you'll explore a variety of the network protocols unique to a Microsoft Windows or Windows-compatible environment. Significant time will be spent exploring the SMB protocol, used for file transfers and countless other purposes in a Microsoft Windows domain structure. Attackers frequently use these protocols to "live off the land" within the victim's environment. By using existing and expected protocols, the adversary can hide in plain sight and avoid deploying malware that could tip off the investigators to their presence and actions.

Exercises

• Visual NetFlow Analysis with SOF-ELK®
• Tracking Lateral Movement with NetFlow
• nfcapd Data Consolidation and Reduction (Bonus lab)
• SMB Session Analysis & Reconstruction

Topics

• NetFlow Collection and Analysis
  • Origins and evolution
  • NetFlow v5 and v9 protocols
  • Architectural components
  • NetFlow artifacts useful for examining encrypted traffic
• Open-Source Flow Tools
  • Using open-source tool sets to examine NetFlow data
  • nfcapd, nfpcapd, and nfdump
  • SOF-ELK®: NetFlow ingestion and dashboards
• File Transfer Protocol (FTP)
  • History and current use
  • Shortcomings in today's networks
  • Capture and analysis
  • File extraction
• Microsoft Protocols
  • Architecture and capture positioning
  • Exchange/Outlook
  • Server Message Block (SMB)

Overview

Commercial tools are an important part of a network forensicator's toolkit. We'll discuss the benefits specific commercial tools may provide, as well as how they may best be integrated into an investigative workflow. With the runaway adoption of wireless networking, investigators must also be prepared to address the unique challenges this technology brings to the table. However, regardless of the protocol being examined or budget used to perform the analysis, having a means of exploring full-packet capture is a necessity, and having a toolkit to perform this at scale is critical.

Commercial tools hold clear advantages in some situations a forensicator may typically encounter. Most commonly, this centers on scalability. Many open-source tools are designed for tactical or small-scale use. Whether using them for large-scale deployments or for specific niche functionalities, these tools can immediately address many investigative needs.

Additionally, we will address the forensic aspects of wireless networking. We will cover similarities with and differences from traditional wired network examinations, as well as what interesting artifacts can be recovered from wireless protocol fields. Some inherent weaknesses of wireless deployments will also be covered, including how attackers can leverage those weaknesses during an attack and how they can be detected.

Finally, we will look at methods that can improve at-scale hunting from full-packet captures, even without commercial tooling. We will look at the open-source Arkime platform and how it can be used in live and forensic workflows. You'll receive a ready-to-use Arkime virtual machine and load source data from an incident we previously investigated, seeking additional clarity from the previously captured full-packet data.

Exercises

• Automated Extraction with NetworkMiner
• Scaling With pcap_iterator.sh
• Using Command-Line Tools for Analysis (Bonus lab)
• Network Forensic Analysis Using Arkime

Topics

• Simple Mail Transfer Protocol (SMTP)
  • Lifecycle of an email message
  • Artifacts embedded along the delivery pathway
  • Adaptations and extensions
• Object Extraction with NetworkMiner
  • Value of commercial tools in a DFIR workflow
  • NetworkMiner
    • Capabilities and user interface
    • Use cases for object extraction
    • Limitations and mitigations
• Wireless Network Forensics
  • Translating analysis of wired networks to the wireless domain
  • Capture methodologies: hardware and software
  • Useful protocol fields
  • Typical attack methodologies based on protection mechanisms
• Automated Tools and Libraries
  • Common tools that can facilitate large-scale analysis and repeatable workflows
  • Libraries that can be linked to custom tools and solutions
  • Chaining tools together effectively
• Full-Packet Hunting with Arkime
  • Arkime's architecture and use cases
  • Methods of ingesting packet data for DFIR workflows
  • Session awareness, filtering, typical forensic use cases
  • Raw packet searching with hunt jobs
  • Enrichment of extracted metadata
  • Custom decoding with CyberChef

Overview

Advancements in common technology have made it easier to be a bad actor and harder for us to track them. Strong encryption methods are readily available and custom protocols are easy to develop and employ. Despite this, there are still weaknesses in the methods of even the most advanced adversaries. As we learn what the attackers have deliberately hidden from us, we must operate carefully to avoid tipping our hats regarding the investigative progress - or the attacker can quickly pivot, nullifying our progress.

Encryption is frequently cited as the most significant hurdle to effective network forensics - for good reason. When properly implemented, encryption can be a brick wall between an investigator and critical answers. However, technical and implementation weaknesses can be used to our advantage. Even in the absence of these weaknesses, the right analytic approach to encrypted network traffic can still yield valuable information about the content. We will discuss the basics of encryption and how to approach it during an investigation. The section will also cover flow analysis to characterize encrypted conversations.

We will also discuss undocumented protocols and the misuse of existing protocols for nefarious purposes. Specifically, we will address how to derive intelligence value with limited or nonexistent knowledge of the carrier protocol.

Finally, we will look at how common missteps can provide the attacker with clear insight to the forensicator's progress. This often leads to the attacker changing their tactics, confounding the investigator, and even erasing all the progress made to that point. We'll address best practices on conducting investigations in a compromised environment and ways to share hard-earned intelligence that mitigate the risks involved.

Exercises

• TLS Profiling
• Decrypting Forward Secrecy and Exploring HTTP/2
• Undocumented Protocol Features
• Mini-Comprehensive Investigation (Bonus lab)
• Capstone Evidence Preparation (Bonus lab)

Topics

• Encoding, Encryption, and SSL/TLS
  • Encoding algorithms
  • Encryption algorithms
    • Symmetric
    • Asymmetric
  • Profiling SSL/TLS connections with useful negotiation fields
  • Analytic mitigation
  • Perfect forward secrecy
• Meddler-in-the-Middle (MITM)
  • Malicious uses and their artifacts
  • Benevolent uses and associated limitations
  • Common MITM tools
• Network Protocol Reverse Engineering
  • Using known protocol fields to dissect unknown underlying protocols
  • Pattern recognition for common encoding algorithms
  • Addressing undocumented binary protocols
  • Follow-on steps in an incident response context
• Investigation OPSEC and Threat Intel
  • Operational Security
    • Basic analysis can tip off attackers
    • How to mitigate risk without compromising quality
  • Intelligence
    • Plan to share smartly
    • Protect intelligence to mitigate risks
• Capstone Challenge Kickoff

Overview

This section will combine all of what you have learned prior to and during this week. In groups, you will examine network evidence from a real-world compromise by an advanced attacker. Each group will independently analyze data, form and develop hypotheses, and present findings. No evidence from endpoint systems is available - only the network and its infrastructure.

Students will test their understanding of network evidence and their ability to articulate and support hypotheses through presentations made to the instructor and class. The audience will include senior-level decision makers, so all presentations must include executive summaries as well as technical details. Time permitting, students should also include recommended steps that could help to prevent, detect, or mitigate a repeat compromise.

Exercises

• Capstone Lab

Topics

• Network Forensic Case
  • Analysis using only network-based evidence
    • Determine the original source of an advanced attacker's compromise
    • Identify the attacker's actions while in the victim's environment
    • Confirm what data the attacker stole from the victim
  • Reporting
    • Present executive-level summaries of your findings at the end of the day-long lab
    • Document and provide low-level technical backup for findings
    • Establish and present a timeline of the attacker's activities