In this blog, we will review how allowing vendor access to your cloud data can lead to compromise, how this has already happened through a vulnerability discovered by SANS Faculty, and how to fix the problem.
Identity and Access Management (IAM) is the primary way to enforce the principle of least privilege in cloud environments. Unfortunately, creating IAM policies is hard and error prone. This makes it difficult to follow the principle consistently.
One rarely discussed area of IAM is integration with third-party systems, such as those provided by vendors. Organizations should distrust external services at least as much, if not more so, than their internal systems and users. These services are vulnerable to supply-chain attacks like all other software. If they have a vulnerability, are injected with malware, or are otherwise compromised, this can compromise the organization that uses them.
As such, organizations should view all vendors as potentially malicious. They should then ask, “What can the vendor do with the access that we are granting them if they wanted to hurt us?” With that threat model in-mind, organizations should grant these vendors as little access as possible while still allowing them to function properly. For these services, such as cross-cloud Cloud Security Posture Management (CSPM) platforms, to work, organizations need to trust them to some extent. Still, like any other system, we should minimize the level of trust and periodically verify that our trust is not abused. This article provides four key requirements for accomplishing this.
This advice applies to (at least) the Big 3 cloud providers (Amazon Web Services (AWS), Microsoft Azure, and Google Cloud). If a vendor wants your cloud data, they must support these requirements. If they do not, your organization’s security will suffer.
High-Level Guidelines for Cloud Vendor Integrations
1. Do not give vendor services unnecessary IAM permissions.
A vendor should only need to perform a small set of operations within a customer’s cloud account. These operations should be codified with IAM policy. For example, a CSPM platform is designed to analyze a cloud account and highlight misconfigurations that impact security. Unless this platform automatically remediates these issues, there is no reason why it should need to modify the account in any way. So, it should only be allowed to perform read-only operations.
Additionally, the CSPM platform should only need to analyze the cloud account’s configuration. It should not be technically capable of reading the organization’s actual data. Even if the vendor only has good intentions, it is possible that they will be compromised. If this happens, the less permissions they have, the better.
2. Use workload identities, not long-lived credentials.
Credentials should live for as short a time as possible. If short-lived credentials are compromised, your organization is compromised. However, this compromise might be temporary. If the attacker was unable to establish persistent access and is not able to generate another set of credentials, then no additional damage can be done.
The Big 3 cloud providers make it fairly easy to create short-lived, automatically rotating credentials for its workloads. This is usually accomplished with a service called the Instance Metadata Service (IMDS). This works great for accessing intra-account resources, but this alone will not work for inter-account access.
As such, many multicloud integrations use long-lived credentials. These can be used until a cloud administrator explicitly deactivates them. They also frequently end up in source code, public software packages, and other places they should not be. The 2023 Datadog State of Cloud Security reported that over 50% of AWS access keys live for more than one year, with a significant number living for over five years. These numbers are slightly smaller for Google and substantially smaller for Azure, but they are still quite significant. Worse yet, these credentials are rarely decommissioned. The same report indicated that nearly 50% of AWS IAM users had unused access keys as of October of 2023.
Thankfully, you can use Workload Identity instead. Eric Johnson, co-author of my course SEC510: Cloud Security Controls and Mitigations, and I have talked about this concept many times in the past, including in this webcast. In short, it allows an Identity Provider (IdP) to trade one of its temporary tokens for a set of temporary credentials used by the target cloud provider. While this is much harder than the alternative, it is also unquestionably the most secure way to perform cross-account authentication. So, your cloud vendor should support the ability to access your cloud resources using workload identity, not long-lived credentials.
3. Prevent the vendor’s other customers from reusing your trust relationship.
By granting the cloud vendor the ability to access your cloud data, you are deputizing them with extensive privileges. Unless the vendor’s service is single-tenant, it will have permission to access cloud accounts used by other customers. It should not be possible for one of the vendor’s customers to access another’s data using these privileges. This is ultimately the vendor’s responsibility, but there are actions that organizations can take to minimize the likelihood of this happening.
Failing to prevent this cross-customer access creates a confused deputy problem. AWS describes this as an issue that occurs when “an entity that doesn't have permission to perform an action can coerce a more-privileged entity to perform the action.” In this context, the “more-privileged” entity is the vendor’s platform, and the “entity that doesn’t have permission” is a malicious user of this platform. It should not be possible for this attacker to trick the platform into accessing another customer’s cloud resources.
AWS documented how a confused deputy problem can occur when organizations grant a third-party access to their resources. In this illustration, they call this third-party vendor “Example Corp.” After an IAM role and trust relationship is created with Example Corp, Example Corp must be provided with the IAM role’s Amazon Resource Name (ARN). Example Corp could then assume that role and access your resources as intended. However, if a different AWS account can trick Example Corp into accessing resources using the same ARN, they might gain unauthorized access to your AWS resources. While AWS’s example is cross-account because the customer and vendor are in two different AWS accounts, this could just as easily be cross-cloud or even on-premises to cloud.
One way to mitigate this issue is to have the vendor supply a customer-specific value when assuming a role. By verifying the value, we can reasonably assume that the request was meant for this customer and not for someone else. AWS shows how this can be done using the ExternalId field when assuming a role. This does not work with the sts:AssumeRoleWithWebIdentity operation. In that case, the assume role policy can verify the RoleSessionId set when performing this operation. Alternative, cloud-agnostic values that can be verified include the aud or sub claims in the IdP’s identity token.
This approach is not foolproof. If the attacker is somehow able to spoof this value, such as by compromising the customer’s account in the vendor’s platform or compromising the platform altogether, this check would be meaningless. So, it is important to vet every vendor’s security posture, especially as it relates to application security.
4. Clean up the required IAM resources after terminating the connection.
Organizations must create various IAM resources for vendors to access their account. These include an identity provider, service account, and trust relationship between the two. The vendor can access the organization’s account for as long as these resources exist. So, if the organization no longer wants to use the vendor, they should decommission these resources. After doing so, even if the vendor’s platform is completely compromised, it would not be able to perform additional operations on the organization’s resources.
Case Study: AWS Security Posture Disclosure through Microsoft Defender for Cloud
On February 7th, 2024, Eric and I reported a confused deputy problem we discovered in Microsoft Defender for Cloud, Azure’s built-in CSPM. One month later, the Microsoft Security Response Center (MSRC) rated this flaw to be “Critical,” and we were awarded a bounty. The vulnerability is very similar to the one described by AWS in the graphic above, but instead of it being cross-account, it is cross-cloud. You can read our full write up on this issue here.
Demonstration and Mitigations
To demonstrate the confused deputy problem, we created a demo application. We jokingly call it the first ever Confused Deputy Insecurity Management (CDIM) platform. It appears to serve the same function as a CSPM, uses another four-letter acronym starting with C, and is 100% free. It is also on the cutting edge. It is the only service in this product category, and Gartner has not even made a Magic Quadrant for CDIM yet!
The CDIM is an AWS-based platform capable of scanning AWS, Azure, and Google Cloud accounts.
In the next section, we will walk through this application and discuss its security flaws. Feel free to follow along with your own AWS account by visiting the platform here.
CDIM Walkthrough
First, we will authenticate using the credentials, “alice” / “hunter2” for Alice, a member of our organization with permissions to deploy infrastructure in our AWS account:
Before performing the scan, Alice must prepare the target AWS account. The CDIM provides her with a Terraform template that can be used to deploy the required IAM resources:
After deploying this template, Alice will enter her AWS account ID and click the scan button:
The CDIM has an... interesting approach to security. Instead of letting you know whether your buckets or storage containers are secure, it simply lists the objects that you have. This seems more like an inventory tool than anything. I guess you get what you pay for!
Can a nefarious outside entity, who we will call Eve, scan Alice’s AWS account? By repeating this scan with Eve’s credentials, (“eve” / “snooping4secrets”), you will see that the answer is yes! Unsurprisingly, the CDIM platform has a confused deputy problem.
How can we fix this issue? We should definitely report the issue to the CDIM’s developers, but we can also take actions on our end to improve the integration. Examine this portion of the AWS Terraform template provided by the CDIM:
This assume role policy ensures only tokens from the CDIM's identity provider with a particular subject can be used to scan Alice’s AWS account. However, there is no condition ensuring the request came from Alice. Meanwhile, the CDIM is assuming this role with a session named after the currently logged in user's username. Observe this snippet from the CDIM's application code:
When Eve performs a scan, the assume role session will be named cdim_eve. So, we can modify the assume role policy to only allow sessions named cdim_alice. The following Terraform block accomplishes this:
There are other issues with the Terraform templates provided by the CDIM. Additionally, the Azure and Google Cloud integrations have security implications as well. We will save these discussions for a future publication. Feel free to continue exploring the platform!
We have discussed a theoretical vulnerability posited by AWS, a real vulnerability in Microsoft Defender for Cloud, and a manufactured vulnerability in our CDIM application demonstration. It is critical to understand these kinds of issues can appear in any cloud integration. Even though the CDIM was designed to be insecure, it still uses workload identity. It is likely that many integrations are even worse and use long-lived credentials. In fact, workload identity has only been generally available in all the Big 3 cloud providers for a few years. We strongly recommend organizations reassess their cloud integrations following the four guidelines above. If you are uncomfortable granting internal users access to your organization’s cloud data, you should be even less comfortable sharing it with vendors or strangers on the internet.
Learn More
This blog post is related to content from SANS Institute’s SEC510: Cloud Security Controls and Mitigations course. SEC510 provides cloud security analysts, engineers, and researchers with practical security controls that can help organizations reduce their attack surface and prevent security incidents from becoming breaches. To learn more, please visit here, review the syllabus, and click the Course Demo button for a free ~1 hour module.
