homepage
Open menu
Contact Sales
Go one level top
  • Train and Certify
    Free Course Demos

    Free course demos allow you to see course content, watch world-class instructors in action, and evaluate course difficulty.

    Train and Certify
    Immediately apply the skills and techniques learned in SANS courses, ranges, and summits
    Learn More
    • Overview
    • Courses
      • Overview
      • Full Course List
      • By Focus Areas
        • Cloud Security
        • Cyber Defense
        • Cybersecurity and IT Essentials
        • DFIR
        • Industrial Control Systems
        • Offensive Operations
        • Management, Legal, and Audit
      • By Skill Levels
        • New to Cyber
        • Essentials
        • Advanced
        • Expert
      • Training Formats
        • OnDemand
        • In-Person
        • Live Online
      • Free Course Demos
    • Training Roadmaps
      • Skills Roadmap
      • Focus Area Job Roles
        • Cyber Defense Job Roles
        • Offensive Operations Job Roles
        • DFIR Job Roles
        • Cloud Job Roles
        • ICS Job Roles
        • Leadership Job Roles
      • NICE Framework
        • Security Provisionals
        • Operate and Maintain
        • Oversee and Govern
        • Protect and Defend
        • Analyze
        • Collect and Operate
        • Investigate
        • Industrial Control Systems
      • European Skills Framework
    • GIAC Certifications
    • Training Events & Summits
      • Events Overview
      • In-Person Event Locations
        • Asia
        • Australia & New Zealand
        • Latin America
        • Mainland Europe
        • Middle East & Africa
        • Scandinavia
        • United Kingdom & Ireland
        • United States & Canada
      • Live Online Events List
      • Summits
    • OnDemand
    • Get Started in Cyber
      • Overview
      • Degree and Certificate Programs
      • Scholarships
      • Free Training & Resources
    • Cyber Ranges
  • Enterprise Solutions
    New Cyber Trends & Training in 2023

    This eBook offers a glimpse into the key threats that are expected to emerge as forecasted by SANS experts.

    Enterprise Solutions

    Build a world-class cyber team with our workforce development programs.

    Learn More
    • Overview
    • Group Purchasing
    • Build Your Team
      • Assessments
      • Private Training
      • By Industry
        • Health Care
        • Industrial Control Systems Security
        • Military
    • Leadership Training
      • Leadership Courses
      • Executive Cybersecurity Exercises
  • Security Awareness
    2023 Security Awareness Report

    Empowering Security Awareness teams with industry benchmarking, program growth, and career development.

    Security Awareness
    Increase your staff’s cyber awareness, help them change their behaviors, and reduce your organizational risk
    Learn More
    • Overview
    • Products & Services
      • Security Awareness Training
        • EndUser Training
        • Phishing Platform
      • Specialized
        • Developer Training
        • ICS Engineer Training
        • NERC CIP Training
        • IT Administrator
      • Risk Assessments
        • Knowledge Assessment
        • Culture Assessment
        • Behavioral Risk Assessment
    • OUCH! Newsletter
    • Career Development
      • Overview
      • Training & Courses
      • Professional Credential
    • Blog
    • Partners
    • Reports & Case Studies
  • Resources
    Security Policy Templates

    In collaboration with security subject-matter experts, SANS has developed a set of security policy templates for your use.

    Resources
    Enhance your skills with access to thousands of free resources, 150+ instructor-developed tools, and the latest cybersecurity news and analysis
    Browse Here
    • Overview
    • Webcasts
      • Webinars
      • Live Streams
        • Wait Just An Infosec
        • Cybersecurity Leadership
        • SANS Threat Analysis Rundown (STAR)
    • Free Cybersecurity Events
      • Free Events Overview
      • Summits
      • Solutions Forums
      • Community Nights
    • Content
      • Newsletters
        • NewsBites
        • @RISK
        • OUCH! Newsletter
      • Blog
      • Podcasts
        • Blueprint
        • Trust Me, I'm Certified
        • Cloud Ace
        • Wait Just an Infosec
      • Summit Presentations
      • Posters & Cheat Sheets
    • Internet Storm Center
    • Research
      • White Papers
      • Security Policies
    • Tools
    • Focus Areas
      • Cyber Defense
      • Cloud Security
      • Digital Forensics & Incident Response
      • Industrial Control Systems
      • Cyber Security Leadership
      • Offensive Operations
      • Open-Source Intelligence (OSINT)
  • Get Involved
    Join the Community

    Membership of the SANS.org Community grants you access to cutting edge cyber security news, training, and free tools that can't be found elsewhere.

    Get Involved
    Help keep the cyber community one step ahead of threats. Join the SANS community or begin your journey of becoming a SANS Certified Instructor today.
    Learn More
    • Overview
    • Join the Community
    • Work Study
    • Teach for SANS
    • CISO Network
    • Partnerships
    • Sponsorship Opportunities
  • About
    Our Mission

    To empower current and future cybersecurity practitioners around the world with immediately useful knowledge and capabilities, we deliver industry-leading community programs, resources and training.

    About
    Learn more about how SANS empowers and educates current and future cybersecurity practitioners with knowledge and skills
    Learn More
    • SANS
      • Overview
      • Our Founder
      • Awards
    • Instructors
      • Our Instructors
      • Full Instructor List
    • Mission
      • Our Mission
      • Diversity
      • Scholarships
    • Contact
      • Contact Customer Service
      • Contact Sales
      • Press & Media Enquiries
    • Frequent Asked Questions
    • Customer Reviews
    • Press
    • Careers
  • SANS Sites
    • Australia
    • Brazil
    • France
    • India
    • Japan
    • Middle East & Africa
    • United Kingdom
  • Log In
  • Join
    • Account Dashboard
    • Log Out
  • Contact Sales
  1. Home >
  2. Blog >
  3. Microsoft 365 Cloud Log Extraction
Megan_Roddie_370x370.png
Megan Roddie

Microsoft 365 Cloud Log Extraction

In this blog post, we discuss the various methods of accessing and exporting the Unified Audit Log (UAL)

July 22, 2023

So far in our blog post series on cloud log extraction, we have looked at extracting logs from AWS, Google Cloud, Google Workspace and Azure. In the fifth and final installment of this series, we’ll be looking at how we can view and extract logs from Microsoft 365.

This first post will look at Microsoft 365. One of the benefits of Microsoft 365’s logging is that they centralize all logs into the Unified Audit Log (UAL). This single log will contain user and admin activity for Azure AD, Exchange, SharePoint, OneDrive, and more. We’ll specifically look at three ways in which we can query and export the UAL:

  • Microsoft Purview Compliance Portal
  • PowerShell
  • Microsoft 365 Management API

All three options have various strengths and weaknesses, so it’ll all depend on your use case which of the methods you want to leverage.

Microsoft Purview Compliance Portal

Accessing the logs via the Microsoft Purview Compliance Portal is best used in situations where you want to perform a quick, targeted search. It has good built-in search functionality but leveraging the data in the portal is more difficult and the export function has restrictions that make it less than ideal for use. If you do choose this method to extract logs, you can use the following steps:

  1. Log into the Purview compliance portal.
  2. Select Audit from the menu on the left hand side.
  3. Run a search by specifying the date range, optionally applying filters, and selecting Search.
    1. Underneath the search options, a list of previous and in progress searches will appear. When your search is complete you will see a Job Status of Completed

  4. Click on the relevant search and on the subsequent page select Export on the top left.
    1. The export may take a moment to complete, especially if the results include a large number of records. You may need to refresh the page to check for completion. When the results are ready, a green status message with a link to the download will be at the top of the screen.

Unfortunately, another drawback of this method is that the SOF-ELK Microsoft 365 parser does not support Purview exports. For SOF-ELK compatibility, you should use the PowerShell method discussed in the next section. The most important data in the CSV export is stored in the form of a JSON blob within the “AuditData” column. As such, if you are going to review the logs in Excel or another CSV viewer, you need to transform that column. Microsoft provides step-by-step instructions on how to do this in their documentation.

PowerShell

PowerShell has a Cmdlet called Search-UnifiedAuditLog that allows you to query and export the UAL via a PowerShell interface. This method is best used when trying to export a small amount of the UAL as it limits the number of records to 5,000. If you need more than 5,000 records and want to leverage PowerShell, we recommend taking a look at the Microsoft Extractor Suite mentioned in the “Open-source UAL Collection Tools” section below.

Before attempting to export the UAL via PowerShell, you need to install the Exchange Online PowerShell module which provides the cmdlets for interacting with Microsoft 365. Furthermore, you need to ensure you have the permissions required to access the UAL via PowerShell. At minimum, you need the View-Only Audit Logs role assigned to your user via a role group in the Microsoft 365 admin center. Once these prerequisites are in place, the following steps can be performed to export the logs.

  1. Connect to Microsoft 365:

    Connect-ExchangeOnline –UserPrincipalName <UPN> -ShowProgress $true

  2. Perform a search, extract the AuditData property, and pipe the output to a UTF8-encoded JSON file.

    Search-UnifiedAuditLog –StartDate 2023-06-01 –EndDate 2023-07-01 -ResultSize 5000 ReturnNextPreviewPage | Select-Object –ExpandProperty AuditData | Out-File –Encoding UTF8 “20230601_ual.json”

    Note: Additional filters can be applied. See the Search-UnifiedAuditLog cmdlet documentation for more parameters.

From this point, you can import the JSON file into SOF-ELK (by copying it to the Microsoft 365 Logstash directory) or another parsing tool of your choosing.

Microsoft 365 Management API

The final method that can be used for exporting the UAL is the Microsoft 365 Management API. This is best for environments looking to continuously query the UAL in order to ingest the logs into an external platform such as a SIEM or large environments that exceed the export limits of the PowerShell Cmdlet. The API endpoint will only return results from the last 7 days, so it must be polled on a continuous basis via a custom-developed application or commercial solution that leverages the API. The high-level steps for using the API are as follows:

  1. Register and configure an application in Azure AD
  2. Generate a new key and X.509 certificate for the application
  3. Authorize Microsoft 365 Management API access
  4. Request access token from Azure AD
  5. Start a subscription to the required log
  6. Retrieve blobs of logs at regular intervals

For more in-depth guidance on leveraging the API, see Microsoft’s documentation here: https://for509.com/ualmgmtapi. If you are leveraging a commercial product for collecting these logs, they will most likely have a built-in integration and configuration guide for connecting your solution to the API.

Open-source UAL Collection Tools

As mentioned in the previous blog post in this series, Invictus IR has released a suite of scripts for extracting Microsoft logs (both 365 and Azure) using PowerShell. One of the major benefits of these scripts is that it breaks down the extraction into chunks in order to avoid the record limit imposed by the Search-UnifiedAuditLog PowerShell cmdlet. The scripts related to extracting the UAL are:

  • Get all the UAL entries. Supports filtering by date and userid only.
  • Get the selected group UAL entries based on pre-defined groups. These groups attempt to compensate for the fact that the Search-UnifiedAuditLog cmdlet doesn’t support filtering by workload.
  • Get specific records by filtering on RecordType in addition to date and userid.
  • Display the total number of logs within the UAL given a date and userid filter.

These scripts can be accessed at no cost at the following Github link: https://for509.com/invictus-suite.

Conclusion

In this blog post, we discussed the various methods of accessing and exporting the Unified Audit Log (UAL). If you need a way to perform a quick and targeted search, you can leverage the Purview Compliance Portal. If you want to retrieve a small part of the UAL and export to JSON format, PowerShell has the Search-UnifiedAuditLog cmdlet. For larger organizations or those who need to continuously poll the UAL, there is the Microsoft 365 Management API.

We hope that this series of blog posts on extracting cloud logs will better prepare you for when an incident occurs in your cloud environment! If you want to learn more about leveraging these logs for incident response, check out SANS FOR509: Enterprise Cloud Forensics and Incident Response.

Share:
TwitterLinkedInFacebook
Copy url Url was copied to clipboard
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Tags:
  • Digital Forensics, Incident Response & Threat Hunting

Related Content

Blog
FOR509.png
Digital Forensics, Incident Response & Threat Hunting
September 8, 2023
FOR509 – Always Evolving
With a significant amount of new content and labs in this release, it’s by far biggest update since last year
Megan_Roddie_370x370.png
Megan Roddie
read more
Blog
Cloud_Blog.png
Digital Forensics, Incident Response & Threat Hunting
September 4, 2023
Evolution of Cloud Tactics, Techniques, and Procedures
This blog post discussed techniques implemented across the kill chain, from initial access to lateral movement to impact.
Megan_Roddie_370x370.png
Megan Roddie
read more
Blog
Hope_for_the_Best_Prepare_for_the_Worst_How_to_prepare_for_cloud_DFIR.png
Digital Forensics, Incident Response & Threat Hunting
August 19, 2023
Hope for the Best, Prepare for the Worst: How to prepare for cloud DFIR
Understand the specific steps that can be taken to significantly improve your organization's cloud incident response efficiency and efficacy.
Megan_Roddie_370x370.png
Megan Roddie
read more
  • Register to Learn
  • Courses
  • Certifications
  • Degree Programs
  • Cyber Ranges
  • Job Tools
  • Security Policy Project
  • Posters & Cheat Sheets
  • White Papers
  • Focus Areas
  • Cyber Defense
  • Cloud Security
  • Cybersecurity Leadership
  • Digital Forensics
  • Industrial Control Systems
  • Offensive Operations
Subscribe to SANS Newsletters
Receive curated news, vulnerabilities, & security awareness tips
United States
Canada
United Kingdom
Spain
Belgium
Denmark
Norway
Netherlands
Australia
India
Japan
Singapore
Afghanistan
Aland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antarctica
Antigua and Barbuda
Argentina
Armenia
Aruba
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius, and Saba
Bosnia And Herzegovina
Botswana
Bouvet Island
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Cook Islands
Costa Rica
Croatia (Local Name: Hrvatska)
Curacao
Cyprus
Czech Republic
Democratic Republic of the Congo
Djibouti
Dominica
Dominican Republic
East Timor
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands (Malvinas)
Faroe Islands
Fiji
Finland
France
French Guiana
French Polynesia
French Southern Territories
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Heard And McDonald Islands
Honduras
Hong Kong
Hungary
Iceland
Indonesia
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Korea, Republic Of
Kosovo
Kuwait
Kyrgyzstan
Lao People's Democratic Republic
Latvia
Lebanon
Lesotho
Liberia
Liechtenstein
Lithuania
Luxembourg
Macau
Macedonia
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Micronesia, Federated States Of
Moldova, Republic Of
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands Antilles
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
Northern Mariana Islands
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Pitcairn
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russian Federation
Rwanda
Saint Bartholemy
Saint Kitts And Nevis
Saint Lucia
Saint Martin
Saint Vincent And The Grenadines
Samoa
San Marino
Sao Tome And Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Sint Maarten
Slovakia
Slovenia
Solomon Islands
South Africa
South Georgia and the South Sandwich Islands
South Sudan
Sri Lanka
St. Helena
St. Pierre And Miquelon
Suriname
Svalbard And Jan Mayen Islands
Swaziland
Sweden
Switzerland
Taiwan
Tajikistan
Tanzania
Thailand
Togo
Tokelau
Tonga
Trinidad And Tobago
Tunisia
Turkey
Turkmenistan
Turks And Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United States Minor Outlying Islands
Uruguay
Uzbekistan
Vanuatu
Vatican City
Venezuela
Vietnam
Virgin Islands (British)
Virgin Islands (U.S.)
Wallis And Futuna Islands
Western Sahara
Yemen
Yugoslavia
Zambia
Zimbabwe

By providing this information, you agree to the processing of your personal data by SANS as described in our Privacy Policy.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • © 2023 SANS™ Institute
  • Privacy Policy
  • Terms and Conditions
  • Do Not Sell/Share My Personal Information
  • Contact
  • Careers
  • Twitter
  • Facebook
  • Youtube
  • LinkedIn