2023 Global Threat Report from CrowdStrike
Before we dig into cloud vendor Instance Metadata Service (IMDS) capabilities and how they can be abused, let’s take a look at a few snippets of the 2023 Global Threat Report from our friends at CrowdStrike. First, did we learn our lessons about IMDS yet? On page 15, the report states:
Throughout 2022, cloud-conscious actors primarily obtained initial access to the cloud by using existing, valid accounts, resetting passwords or placing webshells or reverse shells for persistence after exploiting public-facing applications such as web servers. Once on a machine, actors attempted to gain access primarily through credentials found in files, but also via the cloud provider's instance metadata services (IMDSs).1
So no, we, as a cloud community, did not learn our lessons. IMDS abuse is still a problem and, if we don’t know how to manage this capability properly, attackers can ransack our cloud infrastructure with ease. Just take a look at this snippet from the CrowdStrike report in reference to an adversary group they call PANDA:
After achieving interactivity on the host, the suspected PANDA adversary was observed using the awscli utility to perform advanced reconnaissance, including enumeration of config and credentials files.1
It appears, based on their findings, that it’s not just your operations and security teams that are aware of the various methods to acquire and use cloud credentials to access your cloud account.
Let’s take a moment and unpack IMDS, how it’s used by adversaries to steal credentials information, and some recommendations to better defend against these attacks.
What is IMDS?
When cloud instances/virtual machines in Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), and Oracle Cloud require access to data about itself or the cloud environment, it can query its Instance Metadata Service (IMDS) that typically listens on the IPv4 address of 169.254.169.254 as well as, in the case of AWS, the IPv6 address of fd00:ec2::254. But what kind of information can be had? It depends on the vendor, but typically things like:
- What region and availability zone the instance/VM is running in
- What subnet the instance/VM is a part of
- The image used to launch the system
- The security groups that are used to control network access to the system
- Which public SSH key was injected into the authorized_keys file when spawned
The IMDS endpoints are often in a tree-like structure that you can drill into to retrieve the data of interest. For example, if you were looking for the current AWS security group names tied to this instance, you can issue the following curl command in the Linux instance:
You may think, “what’s the big deal?”, but there are some more sensitive items that can be retrieved as well, like:
- User-data passed to the system at boot time (could contain secrets)
- IAM role credentials (could allow access to the greater AWS cloud account)
- Managed identity credentials (could allow access to the Azure account)
- Service account tokens (allowing access to the GCP account)
We will focus on AWS specifically since it has a default deployment that is vulnerable to a specific attack vector (as you will see shortly). For example, this pair of curl commands can retrieve not only the name of the role name attached to the EC2 instance, but also the current credentials that can be used by the instance (and attacker) to access parts of AWS:
Now, when we say access to the above cloud accounts, that means that the attacker would be able to access the cloud environment with the exact permissions tied to the role, identity, or service account. This is where an attack abusing IMDS is more likely to focus.
How can attackers leverage it?
Theft
Up to this point, you may be assuming that, to get access to IMDS, you need to have a shell session on the cloud-based system, but that is certainly not the only way to access IMDS. Attacks like the Capital One breach in 20192 can leverage a flaw in the virtual system’s software to “trick” the application into querying IMDS instead of retrieving a legitimate resource. This is an example of a Server-Side Request Forgery.
Here is an example code snippet that would be susceptible to an SSRF attack which would allow an attacker to gain access to IMDS:
This seemingly harmless code block (which will simply provide source code of the requested URL sent in as a url GET variable value) will allow an attacker to retrieve IMDS data relatively easily. The attacker could craft a request to the URI of ?url=http%3A%2F%2F169.254.169.254%2Flatest%2Fmeta-data%2Fiam%2Fsecurity-credentials on this web application as shown here:
Since the attacker now knows the name of the attached instance role, the next request would just tack on the name of the role. This request URI would look like: ?url=http%3A%2F%2F169.254.169.254%2Flatest%2Fmeta-data%2Fiam%2Fsecurity-credentials/EC2S3FullAccess.
Oh no! Exposed cloud credentials!
Pivot
Once the attacker has access to the cloud credentials, it’s trivial to pivot into the cloud account and access whatever the instance/VM would normally have access to. Assuming that the attacker has the AWS CLI tools installed on their local system, they could either create an entry in their .aws/credentials file or provide the credentials as environment variables (as shown below).
After the credentials are set, the attacker can then execute AWS commands at will. Some may fail if the instance role does not have the appropriate permissions which may lead to some trial and error. But eventually, the attacker may stumble upon some requests that will work.
Uh oh! An S3 bucket that looks enticing!
And now the crown jewels are gone!
You really... REALLY should not go to that link (you’ve been warned)!
How Can Defenders Protect IMDS?
Latest Versions
Since the above attack’s initial access was due to some vulnerable PHP web code, an obvious place to start would be to fix the code. The web application developer would start by adding input sanitization to the web code (e.g., not allowing 169.254.169.254 to be entered) or adding defense-in-depth measures like a Web Application Firewall (WAF) to thwart these suspicious web requests before they reach the web server.
There is, however, another approach we can take from a cloud configuration perspective. IMDS version 1 is in place and running by default in AWS for most marketplace images (but we are seeing some shifts to enforcing a more secure version as seen in the Amazon Linux 2023 image3). This allows for simple HTTP requests to be used to retrieve information as seen either. However, we can change this by enforcing IMDS version 24 instead. IMDSv2 differs in a few ways:
IMDSv2 Change | Why this change matters |
Requestor must first request a token from IMDS before making any subsequent requests | This changes the attacker’s requirements from two simple GET requests to acquire the role name and role credentials to now needing to first request a token |
When requesting the token, the HTTP method is not a GET, but a PUT | Attacker must somehow change not only the URL being requested, but the HTTP method that is used—this is often times not possible when conducting a SSRF attack |
PUT request must also include an additional HTTP header of X-aws-ec2-metadata-token-ttl-seconds: <seconds> where <seconds> signifies how long the returned token will be valid | Another requirement that the attacker likely does not have the power to influence when conducting a SSRF attack |
A new HTTP header, X-aws-ec2-metadata-token: <TOKEN> must be included (where <TOKEN> is the value of the returned token from the previous request) | Once more, the attacker likely does not have the ability to add HTTP headers to their request during a SSRF attack |
This is what an IMDS version 2 request would look like from the command line:
And here is how to require IMDSv2 on an already deployed EC2 instance (of course, you should be deploying these from the start with only IMDSv2 enabled):
No more SSRF accessing IMDS!
Restricting which local users/groups can submit the request
Now that most SSRF attacks are taken care of, there is still the issue of an exploit which allows remote code execution. In other words, instead of just providing a URL to the web application, the web application is exploited by malicious code—allowing the attacker to execute code of their choosing (like they were on the command-line of that system) or gain shell access to the system and having direct command-line access.
In any case, the attacker can easily run the necessary curl commands to access IMDS—even with version 2 enabled. So... what else can be done?
There is one more trick up our sleeves to add as a defense-in-depth measure: host-based firewall tweaks. In our example, we are using a Linux host as our web server. Since most Linux systems have iptables or similar host-based firewalls installed by default, we can apply a few rules restricting which user account can make an outbound request to 169.254.169.254.
Let’s assume that the only user on the system that should have access to IMDS is the root user to pull the latest and greatest web code from an S3 bucket to this web server (hence the original intention of the instance role). The following iptables rules will prevent all other users from initiating an outbound request to the IMDS service:
The first rule allows any process that is running as the root user to reach 169.254.169.254 and the second rule prevents all other requests to IMDS.
With those rules in place, let’s test them out. Here is a request from a non-root user:
And one from the root user:
If the web application user (which should not be root) is now executing attacker-controlled commands, they should be blocked from accessing IMDS until they are able to somehow either remove the iptables rule or escalate to root. Of course, if the compromised user account is the one that requires access to IMDS, this additional work does us no good here, but would prevent other users from accessing IMDS that have no business doing so.
Takeaway
This information from the CrowdStrike report related to IMDS should be quite concerning as attackers are becoming more and more aware of methods to locate and abuse cloud credentials. As course author for SANS SEC488: Cloud Security Essentials5 and co-author (along with Shaun McCullough) of SEC541: Cloud Security Attacker Techniques, Monitoring, and Threat Detection6, I’ve also found that IMDS is a major blind spot for many organizations’ security teams. If you learned anything from this, hopefully it is to at least migrate all your current and future AWS EC2 instances to IMDS version 2 and to keep a close eye on the usage patterns of credentials that belong to compute resources through your monitoring efforts. Please see the below references for even more details on IMDS.
References
[1] 2023 Global Threat Report. (2023). CrowdStrike. https://www.crowdstrike.com/global-threat-report/
[2] Stella, J. (2019, August 1). A Technical Analysis of the Capital One Cloud Misconfiguration Breach. Fugue. https://www.fugue.co/blog/a-technical-analysis-of-the-capital-one-cloud-misconfiguration-breach
[3] Amazon Web Services. (2023). Comparing Amazon Linux 2 and Amazon Linux 2023. https://docs.aws.amazon.com/linux/al2023/ug/compare-with-al2.html.
[4] Amazon Web Services. (2022). Use IMDSv2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html
[5] SANS Institute. (2023, August 25). SEC488: Cloud Security Essentials. https://www.sans.org/cyber-security-courses/cloud-security-essentials/
[6] SANS Institute. (2023, August 25). SEC541: Cloud Security Attacker Techniques, Monitoring, and Threat Detection. https://www.sans.org/cyber-security-courses/cloud-security-attacker-techniques-monitoring-threat-detection/
ABOUT THE AUTHOR
Ryan's passion for information technology started in 2001 when he found himself constantly trying to make his high school's computers and even calculators do things that they weren't exactly intended to do. They lacked games, so he learned how to create some. Yes, some may call this hacking. Ryan called it "fun", which led to attending college with intentions of becoming a software engineer. During school, Ryan obtained an internship with a very cybersecurity-minded organization -- the Defense Information Systems Agency (DISA). Ever since then, he’s been hooked on cybersecurity. Ryan is the author for SANS SEC488: Cloud Security Essentials, co-author of SEC541: Cloud Security Attacker Techniques, Monitoring, and Threat Detection, and an instructor for SEC530: Defensible Security Architecture and Engineering. Learn more about Ryan here.
