Using Fiddler to inspect Java client library requests

When developing applications that use the Google Cloud API, being able to trace and inspect HTTP requests with a tool like Fiddler can be a great debugging aid. But getting Fiddler to work with the Java client libraries can be a bit tricky.

Suppose we have a Java application that uses the Resource Manager API to do things like listing Google Cloud projects. To do that, the application uses two libraries:

com.google.auth:google-auth-library-oauth2-http for authentication
com.google.apis:google-api-services-cloudresourcemanager for using the Resource Manager API

Let’s see how we can use Fiddler to trace and inspect the HTTP requests of this application.

Configuring Fiddler as HTTP proxy

To inspect our application’s HTTP traffic, we have to configure it to use Fiddler as HTTP proxy. We can do that by adding the following parameter to the Java command line:

-DproxyHost=127.0.0.1 -DproxyPort=8888 -DproxySet=true

These parameters are necessary because Java doesn’t automatically apply the Windows (WinInet) proxy settings.

But running our app with these extra parameters doesn’t give us the result we’re hoping for: Fiddler merely shows a “tunnel to” record…

Missing details

and the app crashes with a SSLHandshakeException:

Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:371)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:309)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
	…
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	…
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	…

To analyze HTTP traffic, Fiddler terminates TLS connection using its own “CA” certificate. Fiddler automatically installs that certificate into the Windows certificate store. But Java doesn’t use that store and therefore doesn’t recognize Fiddler’s certificate as trusted.

Adding Fiddler’s certificate to castore

Instead of using the Windows certificate store, each Java runtime maintains its own certificate store (or trust store in Java lingo) in lib\security\castore. So let’s take Fiddler’s certificate and add it to that castore.

First we need to get the certificate:

Download the Fiddler root certificate from http://localhost:8888/FiddlerRoot.cer

The file is DER-encoded, so we need to convert it to PEM. The easiest way to do this is to just use the GUI.
Double-click the FiddlerRoot.cer file
Click Details > Copy to File
In the Welcome to the certificate export wizard, click Next.
On the Export file format page, select the format Base-64-encoded X.509 (.cer) and click Next.
On the File to export page, pick a file name like FiddlerRoot.pem.cer and click Next.
Click Finish to close the export wizard.

Now let’s add the certificate to the JRE’s cacerts trust store:

keytool -import -noprompt -trustcacerts -alias Fiddler -file FiddlerRoot.pem.cer -cacerts -storepass changeit

Let’s verify that it’s been added:

keytool -list -cacerts -alias Fiddler -storepass changeit

Fiddler, Sep 6, 2022, trustedCertEntry,
Certificate fingerprint (SHA-256): A8:64:3B:CE:0E:E6:B4:FF:0D:0E:D0:5B:9D:7E:04:CE:CC:C4:CC:14:4C:A8:18:4F:89:AB:AA:93:79:CB:E2:A0

That looks good, so let’s run our app again:

Some more details

This looks better: We can see an OAuth request to oauth2.googleapis.com and another call to iamcredentials.googleapis.com (for service account impersonation). But we don’t see any details for the Resource Manager calls…what’s going on there?

Trust store oddities

To make sense of what’s going on, it’s helpful to look at the structure of the Java client libraries:

Libraries

The com.google.auth:google-auth-library-oauth2-http library handles authentication. It’s maintained manually and we can find the sources on GitHub.
The com.google.apis:* are the “classic” libraries for accessing REST APIs. These libraries are auto-generated and fairly lightweight in terms of dependencies and code size.
The com.google.cloud:* are the “newer” libraries which predominantly use gRPC APIs. These libraries are also auto-generated, but significantly more heavyweight than the “classic” ones.

The 3 groups of library behave differently when it comes to TLS:

The com.google.auth:google-auth-library-oauth2-http library behaves like just about any other Java library: It either uses the trust store selected by the javax.net.ssl.trustStore system property or the Java cacerts trust store.

After we updated the castore to contain the Fiddler CA certificate, we were therefore able to trace and inspect the requests made by this library.
The “classic” REST libraries don’t follow that pattern. Instead, they “bring their own” trust store and use that unconditionally. Updating the castore therefore had no effect on our Resource Manager API calls.
The “newer” gRPC libraries also do their own thing, but that’s a different topic.

So how can we get the REST libraries to respect Fiddler’s certificate?

Using a custom Java trust store

Typically, we initialize a REST client with code that looks similar to:

var client = new CloudResourceManager
  .Builder(
    GoogleNetHttpTransport.newTrustedTransport(),  // ← selects the trust store
    new GsonFactory(),
    new HttpCredentialsAdapter(this.credentials))
  .build();

The problematic part of this code snippet is the GoogleNetHttpTransport.newTrustedTransport() call. GoogleNetHttpTransport.newTrustedTransport() creates a NetHttpTransport that uses the library’s embedded trust store. It ignores both the runtime’s castore and the javax.net.ssl.trustStore system property.

But we an replace the GoogleNetHttpTransport with a factory that honors the javax.net.ssl.trustStore system property, just like the com.google.auth:google-auth-library-oauth2-http authentication library:

import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.security.KeyStore;

public class HttpTransport {
  public static NetHttpTransport newTransport() throws GeneralSecurityException, IOException {
    var trustStore = System.getProperty("javax.net.ssl.trustStore");
    var trustStorePassword = System.getProperty("javax.net.ssl.trustStorePassword");
    if (trustStore != null && trustStorePassword != null) {
      //
      // Use a custom keystore.
      //
      try (var trustStoreStream = new FileInputStream(trustStore)) {
        var keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(trustStoreStream, trustStorePassword.toCharArray());
        return new NetHttpTransport
          .Builder()
          .trustCertificates(keyStore)
          .build();
      }
    }
    else {
      //
      // Use the Google keystore.
      //
      return GoogleNetHttpTransport.newTrustedTransport();
    }
  }
}

And use that to initialize the client:

var client = new CloudResourceManager
  .Builder(
    HttpTransport.newTransport(), // <-- !!!
    new GsonFactory(),
    new HttpCredentialsAdapter(this.credentials))
  .build();

If we now launch the JVM without any extra parameters, we get the same behavior as before. But if we set the javax.net.ssl.trustStore and "javax.net.ssl.trustStorePassword system properties, then both the authentication and the REST library use our custom trust store.

Let’s create a new trust store FiddlerRoot.jks and import the Fiddler root certificate:

keytool -import -noprompt -trustcacerts -alias Fiddler -file FiddlerRoot.pem.cer -storepass changeit -keystore FiddlerRoot.jks

We can delete the Fiddler cert from the cacert trust store:

keytool -delete -cacerts -alias Fiddler -storepass changeit

Now all that’s left to do is add two extra parameters to the command line

-DproxySet=true -DproxyHost=127.0.0.1 -DproxyPort=8888 -Djavax.net.ssl.trustStore=c:/path/to/FiddlerRoot.jks -Djavax.net.ssl.trustStorePassword=changeit

And voilà, Fiddler now shows all our requests:

All details

Any opinions expressed on this blog are Johannes' own. Refer to the respective vendor’s product documentation for authoritative information.

« Back to home

Using Fiddler to inspect Java client library requests

Configuring Fiddler as HTTP proxy

Adding Fiddler’s certificate to castore

Trust store oddities

Using a custom Java trust store

Related posts