Google Workspace Log Extraction

In parts 1 and 2 of our series on cloud log extraction, we discussed how to retrieve high-value logs from Google Cloud and AWS. In this post, we are going to take a look at Google Workspace. Google Workspace is unique compared to the other platforms we have discussed as it is a software-as-a-service platform versus the platform-as-a-service and infrastructure-as-a-service model provided by Google Cloud and AWS. Google Workspace can be thought of as Google’s counterpart to Microsoft 365, providing email services and other productivity tools to businesses.

As with the other platforms, we are going to provide a few ways of extracting the logs, each with pros and cons. There are a few approaches we can take for this using the following tools:

• Google Workspace Admin Console
• Google Workspace Reports API
• Google Cloud Logging

While looking at the API, we will first present a high-level explanation of retrieving logs via the API in case you want to create your own scripts tailored to your needs. We will also introduce a couple of open-source tools published by SANS FOR509 Enterprise Cloud Forensics and Incident Response instructors.

Before beginning, it is worth mentioning the retention times of the logs we’ll be working with. Most of the logs have a 6 month retention time, meaning that if you do not export the events within 6 months they will be gone forever. Email log searching is reduced even further to a 30 day window. As such, it's important to have a plan in place to export logs for extended retention as required. More details on retention times, as well as lag times, can be found in Google’s documentation here: https://support.google.com/a/answer/7061566?hl=en.

Extracting Logs via Web Console

The quickest way to extract logs from Google Workspace is via the Workspace Admin console. The logs are located within the Investigation Tool, located under Security > Security Center > Investigation Tool in the Admin console navigation menu or via the following direct link: https://admin.google.com/ac/sc/investigation. The below screenshot shows the initial view of the Investigation Tool.

At this point, we arrive at the first downside to using the Admin console to access logs. Google Workspace does not provide a “single pane of glass” view for logs in their console. Searches can only be done for a single data source (log) at a time. So if you want all activity across the organization, you will have to manually perform the log extraction steps for each log type. We will provide the steps using “Drive log events” as an example, but the process should be repeated for all desired log types.

1. Select the “Data source” you want to get logs for.
2. Add filters or conditions (if applicable).
   1. Multiple filters as well as “and/or” conditions can be leveraged.
   2. For example, if you want to only pull events related to a specific user, you can add an “Actor” filter for that user.
3. Select “Search” and the results will be returned, similar to the following:
   

   
   

4. Select “Export all” from above the search results.
5. Choose your export name and format (Google Sheets or CSV) and select “Export”
6. Once the results are ready for download, they can be obtained via the task list on the top right of the Admin console (represented by an hourglass symbol).
   

Another limitation with this method of export is limits on the total number of results that can be exported. Per Google’s documentation, exports for audit logs are limited to 100,000 rows, although observations indicate at the time of writing, this change is still being rolled out and some logs types may be limited to 10,000 rows. Additionally, email log search results are also still limited to 10,000 rows. If you have an Enterprise Plus or Education Plus license, you can retrieve 30 millions rows for Google Sheets exports, but the limit for CSV exports remains at 100,000 rows.

Between the lack of a view of all events and the export limits, the Investigation Tool is best used for targeted searches and quick information gathering. These limitations minimize the value of using the console for a bulk export and ingest into an external tool less than ideal. For that use case, we recommend the API method discussed below.

Extracting Logs via API

In order to avoid the export limits and be able to retrieve JSON-formatted data that can be parsed by third-party tools, the best method is to use the Reports API. The Reports API is part of Google Cloud’s Admin SDK API and returns the same information that you would find within the Investigation tool described above.

The specific API resource for grabbing the events we’d likely be interested in during incident investigation is the activity resource. The list method is used to query one of the supported applications and either all or a specific user. Query strings and specific fields can be used to filter out a subset of results. The returned payload will provide a list of relevant events with all of the properties for those events in JSON format. By using this endpoint, you can develop a script or integration that can export the logs to a destination of your choice. If you prefer not to write your own code, there are a couple of open-source scripts and tools available to provide this functionality.

Open-Source Tools

Korstiaan Stam, a SANS FOR509 instructor, has developed an open-source tool specifically designed to both extract and provide analysis of Google Workspace logs. This tool, Automated Audit Log Forensic Analysis (ALFA) for Google Workspace, was detailed in a DFIR Summit 2022 talk by Stam: Detecting Malicious Actors in Google Workspace. Not only can it extract the logs, but it’ll also integrate with MITRE in an attempt to identify kill chains based on MITRE event mappings. For a detailed overview of this tool and its usage, watch the aforementioned Summit talk and visit the GitHub repo for ALFA here: https://github.com/invictus-ir/ALFA.

As a secondary option with a more limited feature set, a simple open-source script that will pull a subset of activity logs (those most important to Google Workspace investigations) and put them in a format that can be parsed by SOF-ELK. A recent contribution by a community member added the ability to extract additional logs to the script as well as specify time frames. The script can be found in the FOR509 class GitHub here: https://github.com/dlcowen/sansfor509/tree/main/GWS/gws-log-collection.

The output of either of these scripts can be put into the Google Workspace Logstash folder in SOF-ELK and will be parsed to the gws-* index.

Extracting Logs via Google Cloud Logging

This method is going to be least useful in terms of getting a thorough view of the data because only a certain subset of logs are included when logging to Google Cloud. It is however a  Audit logs for the following services are collected:

• Admin
• Enterprise Group
• Login
• OAuth Token (Enterprise, Education, and Cloud Identity Premium only)
• SAML (Enterprise, Education, and Cloud Identity Premium only)
• Access Transparency (Enterprise and Education only)

The value in using Google Cloud Logging sharing is that it is the simplest way to store logs past the 6 month retention window in Google Workspace and is the only option for automatic forwarding without creating a custom integration via API. Additionally you leverage Google Cloud Logging already, this method may be of use to you regardless. Assuming the default retention settings are used, enabling forwarding to Google Cloud will not incur additional costs. To enable log forwarding to Google Cloud, perform the following steps:

1. Sign in to your Google Admin console with a super administrator account.
2. In the Admin console, go to Menu > Account > Account settings > Legal and compliance.
3. Click Sharing options.
   1. To share data, click Enabled.
4. Click Save.

In order to get the resulting Google Cloud logs exported, you can follow the procedures detailed in part 2 of this blog series, Google Cloud Logging Extraction.

Summary

In this blog post, we reviewed the methods through which we can extract logs from Google Workspace. One option is to query the logs in the Admin Console, although there are several limitations that result in this not being the most ideal method. The best option to obtain the most thorough and easily parsable data is to use the Reports API in the Admin SDK. We also mentioned the use of forwarding logs to Google Cloud for a subset of audit logs to be available via Google Cloud Logging. In the last two blog posts in this series, we’ll look at how to extract logs from Microsoft 365 and Azure.