Authenticating to Google Cloud by using an AWS EC2 instance profile and workload identity federation

To let applications that run on Google Cloud authenticate to Google APIs, we can attach a service account to the underlying compute resource. Applications can then query the metadata server to get temporary credentials, and use these credentials to access the Google APIs they need.

But what if we have an application that runs on AWS and needs access to Google APIs? Attaching a service account obviously won’t work in this case – but we can use workload identity federation to let the application use its AWS credentials to authenticate to Google APIs.

The idea behind Workload identity federation is to set up a trust relationship between Google Cloud and an AWS account that lets applications exchange their AWS credentials against Google credentials by following a four-step process:

Obtain AWS credentials. These can be permanent security credentials or temporary security credentials. For an application running on EC2, the best way to obtain credentials typically is to configure an IAM role and instance profile for the instance so that we don’t have to manage any secrets.
Use the credentials to create a GetCallerIdentity token. As we discussed in a previous post, a GetCallerIdentity token is a JSON document that contains the information that we’d normally include in a request to the AWS GetCallerIdentity endpoint, including a valid request signature.
Pass the GetCallerIdentity token to the Security Token Service (STS) API to get a short-lived Google STS token in return.
Use the STS token to authenticate to the IAM Service Account Credentials API and obtain short-lived Google access tokens for a service account.

The following diagram illustrates the process:

Using the resulting short-lived Google access tokens, the application can then access any Google Cloud resources that the service account has been granted access to.

Client Libraries

In many cases, we don’t actually have to implement this multi-step process ourselves. Instead, we can let the Cloud Client Libraries do it for us by:

Creating a create a credential configuration file that defines where to obtain external credentials from (EC2 IMDS, in this case), which workload identity federation pool and provider to use, and which service account to impersonate
Setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to this file
Using the library to obtain default credentials.

In Python, this looks like:

#!/usr/bin/python3
import google.auth
from google.auth.transport import requests

credentials, project_id = google.auth.default(scopes = "https://www.googleapis.com/auth/cloud-platform")

http_request = requests.Request()
credentials.refresh(http_request)

print("Token: " + credentials.token)

That’s exactly the same code we’d write for an application running on Google Cloud.

Using workload identity federation in .NET

Unfortunately, the C# library lags behind and doesn’t support workload identity federation and credential configuration files yet. But as we did for Azure, we can fill that gap by writing a class that:

Performs the 4-step process described above
Derives from ICredential so that we can use it just like other credentials.

We can save ourselves some work by inheriting from ServiceCredential instead of directly implementing ICredential. ServiceCredential is an abstract base class that implements an access token cache:

To request an access token, ServiceCredential invokes RequestAccessTokenAsync which deriving classes must implement.
ServiceCredential then caches the token and keeps using the token until it’s close to expiry.

So all we really have to implement is RequestAccessTokenAsync. In this method, we have to:

Create the GetCallerIdentity token by using the RequestSigner class from an earlier post.
Exchange the GetCallerIdentity token against an STS token.
Impersonate the service account by using the STS token.

To interact the STS and IAM Credentials APIs, we can use the NuGet packages Google.Apis.CloudSecurityToken.v1 and Google.Apis.IAMCredentials.v1.

With that out of the way, here’s the code:

using Google.Apis.Auth.OAuth2;
using Google.Apis.Auth.OAuth2.Responses;
using Google.Apis.Util;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Sts = Google.Apis.CloudSecurityToken.v1;
using StsData = Google.Apis.CloudSecurityToken.v1.Data;
using Iam = Google.Apis.IAMCredentials.v1;
using IamData = Google.Apis.IAMCredentials.v1.Data;
using Google.Apis.Services;
using Newtonsoft.Json;
using System.Net;
using Amazon.Runtime;
using Amazon.Runtime.Internal.Auth;
using Amazon.Util;

namespace CredentialUtils
{
  public class AwsWorkloadIdentityCredential : ServiceCredential
  {
    /// <summary>
    /// AWS credential
    /// </summary>
    public AWSCredentials AwsSourceCredential;

    /// <summary>
    /// Region to use for AWS IAM.
    /// </summary>
    public string AwsRegion { get; }

    /// <summary>
    /// URL of Workload identity federation provider.
    /// </summary>
    public Uri WorkloadIdentityProvider { get; }

    /// <summary>
    /// Scope(s) to acquire access token for.
    /// </summary>
    public IList<string> Scopes { get; }

    /// <summary>
    /// Service account to impersonate.
    /// </summary>
    public string ServiceAccountEmail { get; }

    public AwsWorkloadIdentityCredential(
      AWSCredentials awsCredential,
      string awsRegion,
      Uri workloadIdentityProvider,
      string serviceAccountEmail,
      IEnumerable<string> scopes)
      : base(new ServiceCredential.Initializer(GoogleAuthConsts.TokenUrl))
    {
      Utilities.ThrowIfNull(awsCredential, nameof(awsCredential));
      Utilities.ThrowIfNull(awsRegion, nameof(awsRegion));
      Utilities.ThrowIfNull(workloadIdentityProvider, nameof(workloadIdentityProvider));
      Utilities.ThrowIfNull(serviceAccountEmail, nameof(serviceAccountEmail));
      Utilities.ThrowIfNull(scopes, nameof(scopes));

      this.AwsSourceCredential = awsCredential;
      this.AwsRegion = awsRegion;
      this.WorkloadIdentityProvider = workloadIdentityProvider;
      this.ServiceAccountEmail = serviceAccountEmail;
      this.Scopes = scopes.ToList();
    }

    public AwsWorkloadIdentityCredential(
      AWSCredentials sourceCredential,
      string awsRegion,
      ulong projectNumber,
      string poolId,
      string providerId,
      string serviceAccountEmail,
      IEnumerable<string> scopes)
      : this(
          sourceCredential,
          awsRegion,
          new Uri($"https://iam.googleapis.com/projects/{projectNumber}/locations/" +
              $"global/workloadIdentityPools/{poolId}/providers/{providerId}"),
          serviceAccountEmail,
          scopes)
    {
    }

    private async Task<CallerIdentityToken> CreateCallerIdentityToken(
      CancellationToken cancellationToken)
    {
      var awsCredential = await this.AwsSourceCredential.GetCredentialsAsync();

      //
      // Use endpoint in requested region, see
      // https://docs.aws.amazon.com/general/latest/gr/sts.html.
      //
      var endpoint = new Uri($"https://sts.{this.AwsRegion}.amazonaws.com");
      var queryParameters = new Dictionary<string, string>
      {
        { "Action", "GetCallerIdentity" },
        { "Version", "2011-06-15" }
      };

      var headers = new Dictionary<string, string>()
      {
        { "x-goog-cloud-target-resource", this.WorkloadIdentityProvider
          .ToString()
          .Replace("https:", string.Empty, StringComparison.OrdinalIgnoreCase) }
      };

      if (!string.IsNullOrEmpty(awsCredential.Token))
      {
        headers["x-amz-security-token"] = awsCredential.Token;
      }

      var signedHeaders = new RequestSigner().CreateHeadersForSignedRequest(
        awsCredential,
        endpoint,
        this.AwsRegion,
        "sts",
        queryParameters,
        headers,
        endpoint.AbsolutePath,
        "POST",
        Array.Empty<byte>());

      var queryString = string.Join(
        "&", 
        queryParameters.Select(kvp => $"{kvp.Key}={kvp.Value}"));

      return new CallerIdentityToken(
        $"{endpoint}?{queryString}",
        "POST",
        signedHeaders);
    }

    /// <summary>
    /// Exchange an AWS credentials against a Google STS token.
    /// </summary>
    private async Task<StsData.GoogleIdentityStsV1ExchangeTokenResponse> ExchangeTokenAsync(
      CallerIdentityToken callerIdentityToken,
      CancellationToken cancellationToken)
    {
      using (var service = new Sts.CloudSecurityTokenService())
      {
        return await service.V1
          .Token(
            new StsData.GoogleIdentityStsV1ExchangeTokenRequest()
            {
              Audience = this.WorkloadIdentityProvider
                .ToString()
                .Replace("https:", string.Empty, StringComparison.OrdinalIgnoreCase),
              GrantType = "urn:ietf:params:oauth:grant-type:token-exchange",
              RequestedTokenType = "urn:ietf:params:oauth:token-type:access_token",
              Scope = string.Join(" ", this.Scopes),
              SubjectTokenType = "urn:ietf:params:aws:token-type:aws4_request",
              SubjectToken = WebUtility.UrlEncode(JsonConvert.SerializeObject(callerIdentityToken))
            })
          .ExecuteAsync(cancellationToken)
          .ConfigureAwait(false);
      }
    }

    /// <summary>
    /// Obtain a (short-lived) access token by impersonating a
    /// service account.
    /// </summary>
    private async Task<IamData.GenerateAccessTokenResponse> ImpersonateServiceAccountAsync(
      string stsToken,
      CancellationToken cancellationToken)
    {
      using (var service = new Iam.IAMCredentialsService(
        new BaseClientService.Initializer()
        {
          //
          // Use the STS token like an access token to authenticate
          // requests.
          //
          HttpClientInitializer = GoogleCredential.FromAccessToken(stsToken)
        }))
      {
        return await service.Projects.ServiceAccounts.GenerateAccessToken(
            new IamData.GenerateAccessTokenRequest()
            {
              Scope = this.Scopes
            },
            $"projects/-/serviceAccounts/{this.ServiceAccountEmail}")
          .ExecuteAsync(cancellationToken)
          .ConfigureAwait(false);
      }
    }

    /// <summary>
    /// Request a new token. The base class invokes this method when
    /// the previous token has expired.
    /// </summary>
    public override async Task<bool> RequestAccessTokenAsync(
      CancellationToken cancellationToken)
    {
      //
      // Create a AWS caller identity token
      //
      var callerIdentityToken = await CreateCallerIdentityToken(cancellationToken);

      //
      // Exchange AWS caller identity token against a Google STS token.
      //
      var stsToken = await ExchangeTokenAsync(
          callerIdentityToken,
          cancellationToken)
        .ConfigureAwait(false);

      //
      // Use STS token to impersonate a service account.
      //
      var serviceAccountToken = await ImpersonateServiceAccountAsync(
        stsToken.AccessToken,
        cancellationToken);

      var timeTillExpiry = ((DateTime)serviceAccountToken.ExpireTime) - this.Clock.UtcNow;

      //
      // Assign resulting access token to property so that it's used until
      // it expires. 
      //
      this.Token = new TokenResponse()
      {
        AccessToken = serviceAccountToken.AccessToken,
        ExpiresInSeconds = (long)timeTillExpiry.TotalSeconds
      };

      return true;
    }

    /// <summary>
    /// Helper class to create request signatures.
    /// </summary>
    private class RequestSigner : AWS4Signer
    {
      /// <summary>
      /// Create signed header for AWS API request using the v4 signature
      /// method. Cf https://docs.amazonaws.cn/en_us/AmazonS3/latest/API/sig-v4-header-based-auth.html.
      /// </summary>
      internal IDictionary<string, string> CreateHeadersForSignedRequest(
        ImmutableCredentials credential,
        Uri endpoint,
        string region,
        string service,
        IEnumerable<KeyValuePair<string, string>> parameters,
        IEnumerable<KeyValuePair<string, string>> headers,
        string resourcePath,
        string httpMethod,
        byte[] body)
      {
        //
        // Take the original headers and add signature headers.
        //

        var enrichedHeaders = new Dictionary<string, string>(headers);
        DateTime signedAt = InitializeHeaders(enrichedHeaders, endpoint);

        var sortedHeaders = SortAndPruneHeaders(enrichedHeaders);
        var bodyHash = AWSSDKUtils.ToHex(
          CryptoUtilFactory.CryptoInstance.ComputeSHA256Hash(
            body),
            true);

        var signature = ComputeSignature(
          credential.AccessKey,
          credential.SecretKey,
          region,
          signedAt,
          service,
          CanonicalizeHeaderNames(sortedHeaders),
          CanonicalizeRequest(
            endpoint,
            resourcePath,
            httpMethod,
            sortedHeaders,
            CanonicalizeQueryParameters(parameters),
            bodyHash),
          null);

        enrichedHeaders["Authorization"] = signature.ForAuthorizationHeader;
        return enrichedHeaders;
      }
    }

    /// <summary>
    /// Helper class to serialize caller identity tokens.
    /// </summary>
    internal class CallerIdentityToken
    {
      public CallerIdentityToken(
        string url,
        string method,
        IEnumerable<KeyValuePair<string, string>> headers)
      {
        this.Url = url;
        this.Method = method;
        this.Headers = headers.Select(h => new CallerIdentityRequestHeader(h));
      }

      [JsonProperty("url")]
      public string Url { get;  }

      [JsonProperty("method")]
      public string Method { get;  }

      [JsonProperty("headers")]
      public IEnumerable<CallerIdentityRequestHeader> Headers { get;  }
    }

    internal class CallerIdentityRequestHeader
    {
      public CallerIdentityRequestHeader(
        KeyValuePair<string, string> keyValuePair)
      {
        this.Key = keyValuePair.Key;
        this.Value = keyValuePair.Value;
      }

      [JsonProperty("key")]
      public string Key { get;  }

      [JsonProperty("value")]
      public string Value { get;  }
    }
  }
}

To run the code, we need an EC2 instance with an appropriate IAM role and instance profile and we have to configure workload identity federation. Once we’ve met these prerequisites, we can use the credential like any other ICredential-based object. For example:

// Obtain AWS credentials
AWSCredentials awsCredential = ... ;

// Create a federated Google credential
var credential = new AwsWorkloadIdentityCredential(
  awsCredential,
  "us-east-1",
  1234567890,                                // Project number
  "my-wi-pool",                              // Pool name
  "my-wi-provider",                          // Provider name
  "[email protected], // Service account to impersonate
  new[] { "https://www.googleapis.com/auth/cloud-platform" });
    
// Use the credential to access Google APIs
var service = var service = new ComputeService(
  new BaseClientService.Initializer()
  {
    HttpClientInitializer = credential
  });

Any opinions expressed on this blog are Johannes' own. Refer to the respective vendor’s product documentation for authoritative information.

« Back to home

Authenticating to Google Cloud by using an AWS EC2 instance profile and workload identity federation

Client Libraries

Using workload identity federation in .NET

Related posts