Application default credentials vs your personal gcloud credentials

gcloud manages two sets of credentials, your personal credentials and application default credentials. Having two separate credentials might seem redundant and can cause surprises the first time you use one of the Google Cloud client libraries. But the two credentials serve different purposes.

Personal gcloud credentials

Before you can use gcloud, you have to authorize the tool by running gcloud auth login and subsequently signing on in a browser window. What happens behind the scenes is that gcloud creates a folder structure in your user profile (that’s %APPDATA%\gcloud on Windows) and populates two files:

The first file, credentials.db, is a SQLite database and is the primary way how gcloud stores the OAuth tokens obtained as part of the initial authorization. The structure of the database is considered private to gcloud and other tools should neither read from the database nor attempt to modify it in any way.

The legacy_credentials folder contains a cleartext copy of the cached user credential structure in a file called adc.json:

{
  "client_id": "...559.apps.googleusercontent.com",
  "client_secret": "Zdoe...",
  "refresh_token": "1//03GEa5...",
  "type": "authorized_user"
}

As the name of the folder suggests, this file is maintained for backwards compatibility.

Once you have authorized gcloud, gcloud will use the saved credentials for all subsequent operations until you run gcloud auth revoke.

Application default credentials

In the last post, we looked at how credentials work in the .NET Google Cloud client library and how they provide an abstraction over service account keys, access tokens, and refresh tokens.

Although the mechanics are slightly different in other language SDKs, there is a common scheme – you usually start by calling GoogleCredential.GetApplicationDefault() (or its equivalent, like google.auth.default() in Python), use the returned credential object in subsequent API calls, and never really have to worry about whether a token might be about to expire or what the right API to refresh a token might be.

But what does GoogleCredential.GetApplicationDefault() and its equivalents actually do? If you have run gcloud auth login before on the same machine, it is tempting to assume that GoogleCredential.GetApplicationDefault() would pick up your saved gcloud credentials (from credentials.db or adc.json) and just work. However, that’s not the case – and if you try anyway, you will see an exception like this:

Unhandled Exception: System.AggregateException: One or more errors occurred. 
---> System.InvalidOperationException: The Application Default Credentials are not available. 
They are available if running in Google Compute Engine. Otherwise, the environment variable 
GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. 

See https://developers.google.com/accounts/docs/application-default-credentials for more information.

So gcloud credentials and application default credentials are obviously not the same thing – but where exactly do the application default credentials come from then?

As it turns out, they can come from a variety of different places and when you call GoogleCredential.GetApplicationDefault(), you initiate a search:

1. First, GoogleCredential.GetApplicationDefault() checks to see if the environment variable GOOGLE_APPLICATION_CREDENTIALS is set. If the variable is set, the API loads the service account file that the variable points to.
2. If the environment variable is not set, the code checks if the file %APPDATA\gcloud\application_default_credentials.json (Windows) or .config\gcloud\application_default_credentials.json (Linux) exists. If it does, it loads credentials from this file instead.
3. If the file is not found, the API checks whether it runs on Compute Engine, Kubernetes Engine, Cloud Run, or App Engine and if it does, it will try to obtain credentials for the service account associated with the VM or application.

If it was not for (2), then it would be fair to say that the notion of application default credentials has nothing to do with gcloud at all. Instead, it is a concept implemented by the client libraries that intends to make it easier for you to load the right credentials depending on the environment the code is executed in.

The reason the paths of application default credentials and gcloud cross in (2) is simply convenience: To make the use and management of application default credentials a little easier, gcloud provides commands to manage an extra set of credentials for you, which the client libraries then considers a candidate for application default credentials.

Once you run gcloud auth application-default login and complete the authorization process in the browser, you will notice that a new file pops up in your gcloud folder – and it is the very file that GoogleCredential.GetApplicationDefault() looks for in step (2):

If you look at the file content, you will also notice that it uses the same JSON format as your personal gcloud credentials in legacy_credentials\adc.json – but crucially, both the Client ID and the token are different. If you paid close attention, you might have also noticed that the consent screen looked different:

The value of application default credentials and using GoogleCredential.GetApplicationDefault() in your code becomes apparent when you develop a server-side application that you plan to later deploy on Google Cloud:

• While developing and testing locally, the code might use the credentials you obtained by running gcloud auth application-default login.
• When running tests on your CI system, the code might use a service account key pointed to by a GOOGLE_APPLICATION_CREDENTIALS environment variable.
• Once deployed to Compute Engine, Kubernetes Engine, or App Engine, the code will use the associated service account configured.

These are three entirely different ways to obtain credentials – yet the code is the same in all three cases.

Takeaway

Application default credentials is a concept implemented by client libraries and they are unrelated to your personal gcloud credentials. When you attempt to load application default credentials, the libraries will look in multiple places for credentials – that makes application default credentials particularly useful for developing server-side applications that you plan to deploy on Google Cloud.

• Share this article: