SEC503: Network Monitoring and Threat Detection In-Depth

Overview

Section 1 begins our bottom-up coverage of the TCP/IP protocol stack, providing deep coverage of TCP/IP to prepare you to better monitor and find threats in your cloud or traditional infrastructure. This is the first step in what we think of as a "Packets as a Second Language" course. After the importance of collecting the packets used in zero-day and other attacks has been established, students are immediately immersed in low-level packet analysis to identify threats and identify TTPs. This section covers the essential foundations such as the TCP/IP communication model, theory of bits, bytes, binary and hexadecimal, and the meaning and expected behavior of every field in the IP header. Students are introduced to the use of open-source Wireshark and tcpdump tools for traffic analysis.

The focus of the material is not on dry memorization of fields and their meaning, but on developing a real understanding of why the headers are defined the way they are and how everything works together. These discussions from the perspective of both attackers and defenders allow students to begin to create threat models to identify both known and unknown (zero-day) behaviors.

All traffic is discussed and displayed using both Wireshark and tcpdump, with the pros and cons of each tool explained and demonstrated. Students can follow along with the instructor viewing the sample traffic capture files supplied. Multiple hands-on exercises after each major topic provide students with the opportunity to reinforce what was just learned. The section ends with hands-on application of all concepts with real-world traffic from an incident in a Bootcamp-style activity.

Full Topic Details

• Concepts of TCP/IP
• Why is it necessary to understand packet headers and data?
• The TCP/IP communications model
• Data encapsulation/de-encapsulation
• Bits, bytes, binary, and hex
• Introduction to Wireshark
• Navigating around Wireshark
• Wireshark profiles
• Examination of Wireshark statistics options
• Stream reassembly
• Finding content in packets
• Network Access/Link Layer: Layer 2
• Introduction to the link layer
• Addressing resolution protocol
• Layer 2 attacks and defenses
• IP Layer: Layer 3
• IPv4 and IPv6
• Examination of fields in theory and practice
• Checksums and their importance, especially for network monitoring and evasion
• Fragmentation: IP header fields involved in fragmentation, composition of the fragments, modern fragmentation attacks
• UNIX Command Line Processing
• Processing packets efficiently
• Parsing and aggregating data to answer questions and research a network
• Using regular expressions for faster analysis

Overview

Section 2 completes the "Packets as a Second Language" portion of this course and lays the foundation for the much deeper discussions to come. Students will gain a deep understanding of the primary transport layer protocols used in the TCP/IP model, in addition to the modern trends that are changing how these protocols are used. We'll explore two essential tools, Wireshark and tcpdump, using advanced features to give you the skills to analyze your own traffic. The focus is on filtering large-scale data down to traffic of interest in order to identify threats in both traditional and cloud-based infrastructure using Wireshark display filters and tcpdump Berkeley Packet Filters. These are used in the context of our exploration of the TCP/IP transport layers covering TCP, UDP and ICMP. Once again, we discuss the meaning and expected function of every header field, covering a number of modern innovations that have very serious implications for modern network monitoring. We analyze traffic not just in theory and function but from the perspective of an attacker and defender, allowing us to expand our threat models of modern TTPs at the network level.

Students can follow along with the instructor viewing the sample capture files supplied. Hands-on exercises after each major topic provide students with the opportunity to reinforce what they just learned. The evening Bootcamp material moves students out of the world of theory and into working through its real-world application. Students learn the practical mechanics of command line data manipulation that is invaluable for packet analysis during an incident and also useful in many other information security and information technology roles. We'll also cover useful techniques to understand what systems are on a cloud or traditional network, how they are communicating, and which services are available without performing active scanning.

Full Topic Details

• Wireshark Display Filters
• Examination of some of the many ways that Wireshark facilitates creating display filters
• Composition of display filters
• Writing BPF Filters
• The ubiquity of BPF and utility of filters
• Format of BPF filters
• Use of bit masking
• TCP
• Examination of fields in theory and practice
• Packet dissection
• Checksums
• Normal and abnormal TCP stimulus and response
• Importance of TCP reassembly for IDS/IPS
• UDP
• Examination of fields in theory and practice
• UDP stimulus and response
• ICMP
• Examination of fields in theory and practice
• When ICMP messages should not be sent
• Use in mapping and reconnaissance
• Normal ICMP
• Malicious ICMP
• QUIC
• Fundamentals
• Examination of fields in theory and practice
• Real-world application: Researching a network
• Who are the top talkers?
• What are people connecting to?
• What services are running on our network?
• What kind of east-west traffic is present?

Overview

Section 3 builds on the foundation of the first two sections of the course, moving into the world of application layer protocols. Using this knowledge, we dive into the state-of-the-art detection mechanisms for threat detection used in cloud, endpoint, hybrid-network, and traditional infrastructure.

The overall focus of the section is on using Snort and/or Suricata and learning to write efficient and effective rules. After introducing some rule-writing basics, the balance of the section introduces more and more features of these threat detection tools while exploring capabilities and deficiencies in the context of some of the most widely used, and sometimes vulnerable, application protocols: DNS, HTTP(S), HTTP2, HTTP3, and Microsoft communications. The focus is on protocol analysis, a key skill in network monitoring, threat detection, and network forensics.

Full Topic Details

• Network Architecture
• Instrumenting the network for traffic collection
• Network monitoring and threat detection deployment strategies
• Hardware to capture traffic
• Introduction to Network Monitoring at Scale
• Function of a network monitoring tools
• The analyst's role in detection
• Analysis flow process
• Introduction to Snort/Suricata
• Configuration of the tools and basic logging
• Writing simple rules
• Using common options
• Effective Snort/Suricata
• More advanced content on writing truly efficient rules for very large networks
• Understanding how to write flexible rules that are not easily bypassed or evaded
• Snort/Suricata "Choose Your Own Adventure" approach to all hands-on activities
• Progressive examination of an evolving exploit, incrementally improving a rule to detect all forms of the attack
• Application of Snort/Suricata to application layer protocols
• DNS
• DNS architecture and function
• DNSSEC
• Modern advances in DNS, such as EDNS (Extended DNS)
• Malicious DNS, including cache poisoning
• Creating rules to identify DNS threat activities
• Encrypted DNS advances
• Microsoft Protocols
• SMB/CIFS
• Detection challenges
• Practical Wireshark application
• Modern HTTP
• Protocol format
• Why and how this protocol is evolving
• Detection challenges
• Changes with HTTP2 and HTTP3
• Real-world Application: Identifying Traffic of Interest
• Finding anomalous application data within large packet repositories
• Extraction of relevant records
• Application research and analysis

Overview

The fundamental knowledge gained from the first three sections provides the foundation for deep discussions of modern and future network intrusion detection systems during Section 4. Everything that students have learned so far is now synthesized and applied to designing optimized threat detection capabilities that go well beyond what is possible with Snort/FirePower/Suricata and next-generation firewalls through the use of advanced behavioral detection using Zeek (or Corelight).

Most of this section is spent introducing Zeek/Corelight, followed by hands-on activities to explore its function and logging capabilities. Basic scripting is introduced, followed by a shift to constructing anomaly-based behavioral detection capabilities using Zeek's scripting language and a cluster-based approach.

After students gain a basic proficiency in the use of Zeek, the instructor will lead them through a practical threat analysis and threat modeling process that is used as the basis for an extremely powerful correlation script to identify any potential phishing activity within a defended network. Further practical will demonstrate how this approach to behavioral analysis and threat modeling is used to fill the gaps in the signature-based detection paradigm used in industry and create zero-day threat detection capabilities for unknown threats.

Students are introduced to the versatile packet crafting tool Scapy, a very powerful Python-based tool that allows for the manipulation, creation, reading and writing of packets. Scapy can be used to craft packets to test the detection capability of any monitoring tool or next-generation firewall. This is especially important when a new user-created network monitoring rule is added, for instance for a recently announced vulnerability. Various practical scenarios and uses for Scapy are provided throughout the course.

The section continues with a discussion of how attackers can evade network monitoring capabilities, including several "zero day" evasion techniques that work against all current network monitoring tools.

This section finishes with skills to extract files and information from packet payloads, the impact of encrypted traffic, and techniques on how to deal with encrypted payloads. The Bootcamp material once again will move students out of theory and into practical use in real-world situations. Students will continue to expand their understanding of the developing incident under analysis in preparation for the final day capstone by applying all the techniques learned so far.

Full Topic Details

• Zeek
• Introduction to Zeek
• Zeek operational modes
• Zeek output logs and how to use them
• Practical threat analysis and threat modeling
• Zeek scripting
• Using Zeek to monitor and correlate related behaviors
• Scapy
• Packet crafting and analysis using Scapy
• Writing packets to the network or a pcap file
• Reading packets from the network or from a pcap file
• Practical Scapy uses for network analysis and network defenders
• Evasion Theory
• Theory and implications of evasions at different protocol layers
• Sampling of evasions
• Necessity for target-based detection
• Zero-day monitoring evasions
• Extract Payloads/Encryption
• Extracting arbitrary application content
• Exporting web and other items from packets
• Encrypted traffic challenges and solutions

Overview

This section continues the trend of less formal instruction and more practical application in hands-on exercises. The section covers three major areas, beginning with data-driven, large-scale analysis and collection using NetFlow and IPFIX. With the deep protocol background developed in the first sections of the course, NetFlow becomes an incredibly powerful tool for performing threat hunting in our cloud and traditional infrastructure. After covering the fundamentals, we'll walk students through more advanced analysis and threat detection using and building custom NetFlow queries. The second area continues the large-scale analysis theme with an introduction to traffic analytics. Various tools and techniques for zero-day threat hunting at the network level are introduced, after which students have the opportunity to put them into practice in hands-on exercises. We'll also discuss and demonstrate cutting-edge applications of artificial intelligence and machine learning techniques for anomaly detection. The final area involves digging into network forensics and incident reconstruction. Students work through three detailed hands-on incidents, utilizing all of the tools and techniques from the entire course.

Full Topic Details

• Using Network Flow Records
• NetFlow and IPFIX metadata analysis
• Using SiLK to find events of interest
• Identification of lateral movement via NetFlow data
• Building custom NetFlow queries
• Threat Hunting and Visualization
• Various approaches to performing network threat hunting at enterprise scale in networks
• Exercises involving approaches to visualizing network behaviors to identify anomalies
• Applications of data science to streamline security operations and perform threat hunting
• Experimenting with an AI-based system to identify network protocol anomalies on a defended network
• Introduction to Network Forensic Analysis
• Theory of network forensics analysis
• Phases of exploitation
• Data-driven analysis versus alert-driven analysis
• Hypothesis-driven visualization

Overview

The course culminates with a hands-on server-based Network Monitoring and Threat Detection capstone that is both fun and challenging. Students compete as solo players or on teams to answer many questions that require using tools and theory covered in the course. The challenge is based on six sections of live-fire real-world data in the context of a time-sensitive incident investigation. It is designed as a "ride-along" event, where students are answering questions based on the analysis that a team of professional analysts performed of these same data.