Group Purchasing
Group Purchasing

FOR710: Reverse-Engineering Malware: Advanced Code Analysis

FOR710Digital Forensics and Incident Response
  • 36 Hours (Self-Paced)
Course authored by:
Anuj Soni
Anuj Soni
FOR710: Reverse-Engineering Malware: Advanced Code Analysis
Course authored by:
Anuj Soni
Anuj Soni
  • 36 CPEs

    Apply your credits to renew your certifications

  • Self-paced

    Train at your own pace from wherever you are

  • Advanced Skill Level

    Course material is geared for cyber security professionals with hands-on experience

  • 12 Hands-On Lab(s)

    Apply what you learn with hands-on exercises and labs

Explore and tackle real-world reverse engineering malware scenarios. Learn to dissect sophisticated Windows executables that dominate headlines and preoccupy incident response teams around the globe.

Course Overview

FOR710: Reverse-Engineering Malware: Advanced Code Analysis prepares malware specialists to dissect sophisticated Windows executables, such as those that dominate the headlines and preoccupy incident response teams across the globe. This malware reverse engineering course not only includes essential background and instructor-led walkthroughs, but also provides students with numerous opportunities to tackle real-world reverse engineering scenarios during class.

As defenders hone their analysis skills and automated malware detection capabilities improve, malware authors have worked harder to achieve execution within the enterprise. The result is malware that is more modular, with multiple layers of obfuscated code that executes in-memory to reduce the likelihood of detection and hinder analysis. Malware analysts must be prepared to tackle these advanced capabilities and use automation whenever possible to handle the volume, variety and complexity of the steady stream of malware targeting the enterprise.

FOR710: Advanced Code Analysis continues where FOR610: Reverse-Engineering Malware: Malware Analysis Tools and Techniques course leaves off, helping students who have already attained intermediate-level malware analysis training to take their reverse-engineering skills to the next level. Authored by SANS Certified Instructor Anuj Soni, this course prepares malware specialists to dissect sophisticated Windows executables, such as those that dominate the headlines and preoccupy incident response teams across the globe.

Developing deep reverse-engineering skills requires consistent practice. This reverse engineering course not only includes the necessary background and instructor-led walkthroughs, but also provides students with numerous opportunities to tackle real-world reverse engineering scenarios during class.

Author Statement

"As malware gets more complicated, malware analysis has as well. In recent years, malware authors have accelerated their production of dangerous, undetected code using creative evasion techniques, robust algorithms, and iterative development to expand on existing weaknesses. Proficient reverse engineers must perform in-depth code analysis and employ automation to peel back the layers of code, characterize high-risk functionality and extract obfuscated indicators."

- Anuj Soni

What You’ll Learn

  • Handle code obfuscation techniques like control flow flattening, string encryption, and steganography
  • Analyze loaders, droppers, payloads, and persistence mechanisms
  • Locate and extract deobfuscated shellcode during runtime
  • Analyze files like documents, images, and archives for embedded malware
  • Understand structures and fields (entry point, sections), and their significance
  • Debug and analyze process data structures (PEB, TIB), and investigate memory usage
  • Utilize Python, DBI, Ghidra, and binary emulation for efficient analysis

Business Takeaways

  • Gain deep visibility into high-impact threats to improve detection and response
  • Boost efficiency and scale malware analysis through automation
  • Produce high-confidence threat intelligence and actionable insights to guide defense

Course Syllabus

Explore the course syllabus below to view the full range of topics covered in FOR710: Reverse-Engineering Malware: Advanced Code Analysis.

Section 1Code Deobfuscation and Execution

Malware authors employ evasion techniques and in-memory execution to hide data, hinder analysis, and thwart detection. First, we discuss the use of malicious steganography. Then we cover key steps in program execution, identifying how code is launched and labeling functions accordingly. Finally, we analyze shellcode with the support of WinDbg Preview.

Topics covered

  • Analyzing code deobfuscation
  • Identifying program execution
  • Understanding shellcode execution

Labs

  • Investigating code deobfuscation
  • Analyzing malicious program execution
  • Analyzing shellcode execution

Full Topic Details

  • Analyzing Code Deobfuscation
    • Common approaches to code obfuscation
    • Steganography approaches
    • Key assembly operations
    • Multi-component malware
    • Windows memory allocation
  • Identifying Program Execution
    • Portable Executable (PE) headers and fields
    • Key steps in program execution
    • Memory-mapped files
    • Entry point identification
  • Understanding Shellcode Execution
    • Identifying and extracting shellcode
    • API hashing
    • The Process Environment Block (PEB) and related structures
    • WinDbg Preview for shellcode analysis

Section 2Encryption in Malware

Section 2 tackles the use of encryption in malware. Adversaries use cryptography for many reasons: to encrypt files, protect keys, conceal configuration settings, and obfuscate command and control (C2) communications. Reverse engineers must be prepared to investigate and articulate the purpose of routines in high-impact malware that implement encryption.

Topics covered

  • Encryption essentials
  • File encryption and key protection
  • Data encryption in malware

Labs

  • Encryption essentials knowledge quiz
  • Identifying file encryption and key protection in ransomware
  • Analyzing data encryption in malware

Full Topic Details

  • Encryption Essentials
    • Use cases for crypto usage in malware
    • Symmetric vs. asymmetric encryption
    • Block vs. stream ciphers
    • Modes of operation
    • Common algorithms in malware
    • Microsoft CryptoAPI
  • File Encryption and Key Protection
    • Identifying algorithms in code
    • Common implementations in malware
    • Locating encryption functions
    • Differentiating similar ciphers
  • Data Encryption in Malware
    • Common uses cases for data encryption in malware
    • Symmetric algorithms used for data protection
    • Identifying the cipher
    • Extracting key information
    • Decrypting data

Section 3Automating Malware Analysis

Section 3 covers automating malware analysis. We introduce Python, writing scripts to decrypt configuration data, deobfuscate strings, and extract payloads. We cover a Dynamic Binary Instrumentation (DBI) framework, injecting and executing code within a process to examine its internals. We write Python scripts to automate debugging and dump unpacked code.

Topics covered

  • Python for malware analysis
  • Dynamic Binary Instrumentation (DBI)

Labs

  • Automating config extraction with Python
  • Automating payload extraction with Frida

Full Topic Details

  • Python for Malware Analysis
    • Introduction to Python programming
    • Visual Studio Code
    • Jupyter Notebooks
    • Modules for PE file analysis
    • Config decryption and extraction
  • Malware Analysis with Dynamic Binary Instrumentation (DBI) Frameworks
    • Introduction to DBI frameworks
    • Using DBI frameworks to automate debugging
    • Writing Frida Python scripts to decrypt data and dump code

Section 4Automating Malware Analysis (Continued)

In this section, we continue discussing approaches to automating malware analysis. We introduce Ghidra’s API and write Python scripts to accelerate static code analysis. We also examine the value of binary emulation frameworks and use the Qiling framework to simulate execution and deobfuscate code and data.

Topics covered

  • Automating Aanalysis within Ghidra
  • Binary emulation frameworks

Labs

  • Scripting with Ghidra
  • Emulating code with Qiling (using Ghidra)
  • Emulating code with Qiling (using SMDA)

Full Topic Details

  • Automating Analysis within Ghidra
    • Flat and Program APIs
    • Python scripting with Ghidra
    • Automating data deobfuscation
  • Binary Emulation Frameworks
    • Emulating code execution with Qiling
    • Ghidra’s headless analyzer
    • SMDA disassembler
    • Implementing hooks
    • YARA rule development and yara-python

Section 5Advanced Malware Analysis Tournament (Extended Access)

Section 5 allows students to flex new skills in an independent competitive environment. With extended access to a capture the flag (CTF) platform, students must recall key concepts and perform workflows discussed in class to succeed. This is an opportunity to analyze real-world, complex malware samples and reinforce your new advanced code analysis skills.

Things You Need To Know

Important! Bring your own system configured according to these instructions.

A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will not be able to fully participate in hands-on exercises in your course. Therefore, please arrive with a system meeting all of the specified requirements.

Back up your system before class. Better yet, use a system without any sensitive/critical data. SANS is not responsible for your system or data.

Mandatory System Hardware Requirements

  • CPU: 64-bit Intel i5/i7 (8th generation or newer), or AMD equivalent. A x64 bit, 2.0+ GHz or newer processor is mandatory for this class.
  • CRITICAL: Apple Silicon devices cannot perform the necessary virtualization and therefore cannot in any way be used for this course.
  • BIOS settings must be set to enable virtualization technology, such as "Intel-VTx" or "AMD-V" extensions. Be absolutely certain you can access your BIOS if it is password protected, in case changes are necessary.
  • 16GB of RAM or more is required.
  • 200GB of free storage space or more is required.
  • At least one available USB 3.0 Type-A port. A Type-C to Type-A adapter may be necessary for newer laptops. Some endpoint protection software prevents the use of USB devices, so test your system with a USB drive before class.
  • Wireless networking (802.11 standard) is required. There is no wired Internet access in the classroom.

Mandatory Host Configuration And Software Requirements

  • Your host operating system must be the latest version of Windows 10, Windows 11, or macOS 10.15.x or newer.
  • Fully update your host operating system prior to the class to ensure you have the right drivers and patches installed.
  • Linux hosts are not supported in the classroom due to their numerous variations. If you choose to use Linux as your host, you are solely responsible for configuring it to work with the course materials and/or VMs.
  • Local Administrator Access is required. (Yes, this is absolutely required. Don't let your IT team tell you otherwise.) If your company will not permit this access for the duration of the course, then you should make arrangements to bring a different laptop.
  • You should ensure that antivirus or endpoint protection software is disabled, fully removed, or that you have the administrative privileges to do so. Many of our courses require full administrative access to the operating system and these products can prevent you from accomplishing the labs.
  • Any filtering of egress traffic may prevent accomplishing the labs in your course. Firewalls should be disabled or you must have the administrative privileges to disable it.
  • Download VMware Workstation Pro 17.5.X+ for Windows hosts or VMWare Fusion Pro 13.5.X+ for macOS hosts prior to class beginning. If you do not own a licensed copy of VMware Workstation Pro or VMware Fusion Pro, you can download a free 30-day trial copy from VMware. VMware will send you a time-limited serial number if you register for the trial at their website. This course requires a "Pro" version of VMware software. The "Player" versions are not sufficient.
  • On Windows hosts, VMware products might not coexist with the Hyper-V hypervisor. For the best experience, ensure VMware can boot a virtual machine. This may require disabling Hyper-V. Instructions for disabling Hyper-V, Device Guard, and Credential Guard are contained in the setup documentation that accompanies your course materials.
  • Download and install 7-Zip (for Windows Hosts) or Keka (for macOS hosts). These tools are also included in your downloaded course materials.

Your course media is delivered via download. The media files for class can be large. Many are in the 40-50GB range, with some over 100GB. You need to allow plenty of time for the download to complete. Internet connections and speed vary greatly and are dependent on many different factors. Therefore, it is not possible to give an estimate of the length of time it will take to download your materials. Please start your course media downloads as soon as you get the link. You will need your course media immediately on the first day of class. Do not wait until the night before class to start downloading these files.

Your course materials include a "Setup Instructions" document that details important steps you must take before you travel to a live class event or start an online class. It may take 30 minutes or more to complete these instructions.

Your class uses an electronic workbook for its lab instructions. In this new environment, a second monitor and/or a tablet device can be useful for keeping class materials visible while you are working on your course's labs.

If you have additional questions about the laptop specifications, please contact customer service.

FOR710 training is recommended for a diverse range of individuals, including:

  • Cyber security professionals who want to improve upon their intermediate-level reverse-engineering skills.
  • Reverse engineers who need to improve their abilities to analyze obfuscated code, assess encryption capabilities in malware, and automate analysis tasks.

  • Windows 10 VM with pre-installed malware analysis and reversing tools.
  • Real-world malware samples to examine during and after class.
  • Coursebooks and workbook with detailed step-by-step exercise instruction.

FOR710 is an advanced level Windows reverse-engineering course that skips over introductory and intermediate malware analysis concepts. This course assumes that students have knowledge and skills equivalent to those discussed in the SANS FOR610 Reverse-Engineering Malware course. Students should have at least six months of experience performing behavioral analysis, dynamic code analysis (i.e., using a debugger), and static code analysis (i.e., analyzing disassembled executable content). In addition, students should have some prior exposure to the Ghidra reverse engineering framework. If you're not familiar with this capability, consider watching this brief introduction by Anuj Soni.

The FOR710 course is a part of the “Digital Forensics, Malware Analysis, & Threat Intelligence” Learning Path, which aims to equip cybersecurity professionals with specialized investigative skills.

Depending on your current or desired future role, one of these courses is a great next step in your cybersecurity journey:

Reverse-engineering malware involves analyzing malicious software to understand its functionality, behavior, and operation. The process includes disassembling the code, debugging its execution, and analyzing the malicious software’s data structures and techniques.

This process of advanced code analysis is crucial for several reasons:

  • Threat intelligence: Reverse-engineering helps security professionals gain insights into new threats, attack patterns, and developing countermeasures
  • Incident response: Analyzing the software helps uncover the root cause of infections, removing malware, and mitigating the impact of attacks
  • Vulnerability research: Identifying software and operating system vulnerabilities exploited by malware allows security teams to bolster the organization’s defenses against future attacks
  • Threat hunting: Reverse-engineering helps organizations detect and recognize malicious activity on networks
  • Antivirus development: The analysis process empowers teams to create signatures and detection rules for antivirus software

While malware reverse engineering courses require specialized tools and training, the skills gained are essential for defenders across every cybersecurity discipline.

FOR710: Reverse-Engineering Malware: Advanced Code Analysis equips you with valuable skills that are highly sought-after in the cybersecurity field. You will gain in-depth knowledge of how malware functions, enabling you to:

  • Identify and analyze emerging threats: Understand attack vectors and develop effective countermeasures  
  • Investigate security incidents: Quickly pinpoint the root cause of breaches and assist in swift recovery  
  • Contribute to vulnerability research: Discover and help patch security flaws exploited by malware  
  • Enhance your technical expertise: Develop a strong foundation in systems, networking, and programming, making you a more valuable asset in any cybersecurity role  

This elite-level malware analysis training will significantly expand your career prospects and earning potential in an increasingly threat-driven world.

Relevant Job Roles

Digital Forensics (DGFS)

Skills Framework for the Information Age

Retrieval and interpretation of data from devices and networks to support incident response, compliance investigations, and legal cases. Work focuses on evidence integrity, validated methods, and attacker attribution.

Explore learning path

Malware Analyst

Digital Forensics and Incident Response

Malware analysts face attackers’ capabilities head-on, ensuring the fastest and most effective response to and containment of a cyber-attack. You look deep inside malicious software to understand the nature of the threat – how it got in, what flaw it exploited, and what it has done, is trying to do, or has the potential to achieve.

Explore learning path

Cyber Incident Responder Training, Salary, and Career Path

European Cybersecurity Skills Framework

Monitor the organisation’s cybersecurity state, handle incidents during cyber-attacks and assure the continued operations of ICT systems.

Explore learning path

Digital Forensics (OPM 212)

NICE: Protection and Defense

Responsible for analyzing digital evidence from computer security incidents to derive useful information in support of system and network vulnerability mitigation.

Explore learning path

Cybercrime Investigator Training, Salary, and Career Path (OPM 221)

NICE: Investigation

Cybercrime Investigators navigate dark web forums, trace cybercriminal activity, and conduct covert investigations. They follow forensic and legal standards to gather evidence and respond to cybercrimes.

Explore learning path

Military Operations / Law Enforcement Agents

Digital Forensics and Incident Response

Execute digital forensic operations under demanding conditions, rapidly extracting critical intelligence from diverse devices. Leverage advanced threat hunting and malware analysis skills to neutralize sophisticated cyber adversaries.

Explore learning path

Intrusion Detection/SOC Analysts

Digital Forensics and Incident Response

Analyze network and endpoint data to swiftly detect threats, conduct forensic investigations, and proactively hunt adversaries across diverse platforms including cloud, mobile, and enterprise systems.

Explore learning path

Incident Response

SCyWF: Protection And Defense

This role investigates, analyzes and responds to cyber incidents. Find the SANS courses that map to the Incident Response SCyWF Work Role.

Explore learning path

Course Schedule and Pricing

Have Questions?Contact Us
  • Location & instructor

    Virtual (OnDemand)

    Instructed by
    Date & Time
    OnDemand (Anytime)Self-Paced, 4 months access
    Course price
    $8,780 USD*Prices exclude applicable local taxes
    Registration Options
Showing 1 of 1

Benefits of Learning with SANS

Bryan Simon: Teacher Standing Next to Smartboard and Explaining Concept

Get feedback from the world’s best cybersecurity experts and instructors

OnDemand Mobile App

Choose how you want to learn - online, on demand, or at our live in-person training events

Close Up of Woman Holding a Pen and Documents

Get access to our range of industry-leading courses and resources