Authenticating to Google Cloud from an AWS Lambda function

Using workload identity federation, we can let an AWS-hosted application authenticate to Google Cloud using its AWS credentials, no service account keys required.

Or, more specifically, we can configure the Google Cloud client library to take the application’s AWS credentials, perform a token exchange, impersonate a Google Cloud service account, and use the service account’s identity to make Google API calls.

Setting up workload identity federation and configuring an application to use it typically involves 4 steps:

1. Setting up a workload identity pool and provider.
2. Creating a service account that the AWS-hosted application can impersonate.
3. Creating a credential configuration file that captures all necessary parameters for the client library to authenticate using workload identity.
4. Configuring the AWS-hosted application to use the credential configuration file.

This process works for both EC2 and Lambda. But there are a few differences.

EC2 vs Lambda

EC2 and Lambda differ in how applications can access AWS credentials: On EC2, applications retrieve their credentials by querying the instance metadata server. On Lambda, the runtime injects credentials into the application’s environment by populating the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN.

This difference is relevant to the Google Cloud client libraries. To perform a token exchange, the library has to prepare a GetCallerIdentity token that it can pass to the Google STS. This token is essentially an unsent GetCallerIdentity request, formatted as a JSON document:

{
  "headers": [
    {
      "key": "host",
      "value": "sts.ap-southeast-1.amazonaws.com"
    },
    {
      "key": "x-amz-date",
      "value": "20231212T233130Z"
    },
    {
      "key": "x-amz-security-token",
      "value": "IQoJb3Jp..."
    },
    {
      "key": "x-goog-cloud-target-resource",
      "value": "//iam.googleapis.com/projects/6716.../locations/global/workloadIdentityPools/..."
    },
    {
      "key": "Authorization",
      "value": "AWS4-HMAC-SHA256 Credential=ASIA..., Signature=c5c79bd5addb57ff9614a71809…"
    },
    {
      "key": "x-goog-cloud-target-resource",
      "value": "//iam.googleapis.com/projects/6716.../locations/global/workloadIdentityPools/..."
    }
  ],
  "method": "POST",
  "url": "https://sts.ap-southeast-1.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
}

Notice that the token doesn’t contain the AWS key id, secret, or token. But it contains a signature, and to create that signature, a client library needs the application’s AWS credentials.

The client libraries for most languages know how to obtain credentials from the instance metadata server and thus support EC2. Some client libraries – notably Java and Python – also support AWS Lambda, while others (such as .NET) lag behind a bit.

But there is another difference between EC2 and Lambda: The standard way to “tell” an application to use workload identity federation is to:

1. Place a credential configuration file somewhere on disk.
2. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to the file.

This approach works well on EC2, but might not be ideal on Lambda: In a serverless environment, a more natural approach to handle configuration is to let the application load it from environment variables.

Let’s see how we can write a Lambda function that uses workload identity federation and loads the necessary parameters from the environment.

Loading credential configuration from the environment

The typical pattern to obtain Google credentials in a Java application is to use the application default credentials mechanism:

GoogleCredentials credentials =
    GoogleCredentials.getApplicationDefault();

To let a Lambda function use workload identity federation and obtain the necessary configuration from the environment, we have to deviate from this pattern and initialize a GoogleCredentials object programmatically instead.

GoogleCredentials is an abstract base class, so we can’t create a GoogleCredentials directly. Instead, we use the AwsCredentials class which is one of the many classes derived from GoogleCredentials:

Initializing an AwsCredentials object requires a number of parameters. Most of the parameters are essentially constants and only 3 are really interesting:

• Audience: encodes the workload identity pool and provider.
• Service account name: the service account to impersonate.
• Region: the AWS region to use for making the GetCallerIdentity call.

For the region, we can use the AWS_DEFAULT_REGION environment variable that Lambda provides by default. For the other two, we introduce new environment variables:

  private static ExternalAccountCredentials authenticateUsingEnvironmentVariables()
  {
    //
    // Read environment variables.
    //
    var audience = System.getenv("GOOGLE_WORKLOADIDENTITY_AUDIENCE");
    var serviceAccount = System.getenv("GOOGLE_WORKLOADIDENTITY_SERVICEACCOUNT");
    var region = System.getenv("AWS_DEFAULT_REGION");
    
    Preconditions.checkNotNull(
      audience, 
      "GOOGLE_WORKLOADIDENTITY_AUDIENCE must be set");
    Preconditions.checkNotNull(
      serviceAccount,
      "GOOGLE_WORKLOADIDENTITY_SERVICEACCOUNT must be set");
    
    //
    // Initialize a credential object.
    //
    return AwsCredentials.newBuilder()
      .setSubjectTokenType("urn:ietf:params:aws:token-type:aws4_request")
      .setTokenUrl("https://sts.googleapis.com/v1/token")
      .setAudience(audience)
      .setCredentialSource(new AwsCredentialSource(Map.of(
        "environment_id",
        "aws1",
        "regional_cred_verification_url",
        String.format("https://sts.%s.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15", region)
      )))
      .setServiceAccountImpersonationUrl(
      String.format(
        "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/%s:generateAccessToken",
        serviceAccount))
      .build();
  }

With this helper function in place, we can replace the snippet above with:

GoogleCredentials credentials = 
    authenticateUsingEnvironmentVariables();

And now we can configure the audience and service account in the AWS console:

For a complete, working example for an AWS Lambda function that uses workload identity federation, see the jpassing/workloadidentity-examples repository.

• Share this article: