You can use libssh2 with CryptoNG… unless you need ECDSA

Libssh2 lets us choose between multiple different crypto backends. But that doesn’t mean these backends are interchangeable.

When we build libssh2, we must make a choice on which crypto backend we want to use. The most common choice is probably OpenSSL, but we can also use libgcrypt, mbedTLS, wolfSSL or WinCNG.

The last option, WinCNG, is interesting for a few reasons:

• It’s specific to Windows because it uses the Windows CryptoNG API (CNG) to perform all cryptographic operations.
• It frees us from having to bundle and ship any third-party crypto library, which means we also don’t have to worry about keeping that library up to date. Crypto NG comes with Windows, and keeping it patched and up to date is Microsoft’s problem.
• It lets us reduce the size of libssh2.dll: Statically linking to OpenSSL results in a libssh2.dll of about 1.8 MB in size. Switching to CNG lets us cut that to about 250 KB.

The prospect of using CNG as the crypto backend was one of the reasons I selected libssh2 for IAP Desktop. Unfortunately, back in 2020, CNG was broken on Windows 10 so that I ended up using OpenSSL instead. But at least the hope remained that some day, I’d be able to switch to CNG.

Unfortunately, that day still hasn’t come yet.

ECDSA woes

Switching from the OpenSSL backend to CryptoNG is pretty straightforward and only requires adding -DCRYPTO_BACKEND=WinCNG to the libssh2 build options.

But running IAP Desktop with a CNG-backed version of libssh2 reveals some surprises. Connecting to VMs and authenticating using ssh-rsa works just fine. But once we switch to ECDSA and try to connect to a VM that doesn’t permit rsa-ssh, things suddenly break:

Looking at the server’s /var/log/secure log, we can see the following log entry:

Jun 18 01:32:56 centos-fips-1 sshd[30672]: Unable to negotiate with 35.235…. port 34657: 
no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth]

Apparently, IAP Desktop isn’t even offering ecdsa-sha2-nistp256… why’s that?

Let’s take a look at IAP Desktop’s own log:

Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.304121 Key Ex: Sent KEX: diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1
Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.304121 Key Ex: Sent HOSTKEY: ssh-rsa,ssh-dss
Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.305118 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,[email protected],aes192-cbc,aes128-cbc,arcfour128,arcfour,3des-cbc
Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.305118 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,[email protected],aes192-cbc,aes128-cbc,arcfour128,arcfour,3des-cbc
Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.306119 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
Google.Solutions.Ssh Verbose: 0 : [libssh2] 93.306119 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96

Indeed, libssh2 only offers ssh-rsa, and ssh-dss. Let’s compare that to a “normal” build of IAP Desktop that uses OpenSSL:

Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent KEX: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,curve25519-sha256,[email protected],diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent HOSTKEY: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa,ssh-dss
Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent CRYPT_CS: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,[email protected],aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent CRYPT_SC: aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,[email protected],aes192-cbc,aes128-cbc,blowfish-cbc,arcfour128,arcfour,cast128-cbc,3des-cbc
Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent MAC_CS: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,[email protected]
Google.Solutions.Ssh Verbose: 0 : [libssh2] 318.779062 Key Ex: Sent MAC_SC: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96,hmac-ripemd160,[email protected]

That looks much different: by switching from OpenSSL to CNG, we’ve lost the capability to use ECDSA and EdDSA!

Missing EdDSA support isn’t a surprise as CryptoNG doesn’t support EdDSA. But CryptoNG does support ECDSA just fine, and not exposing that capability is a limitation of the CNG code in libssh2.

Being limited to ssh-dss (which is outdated) and ssh-rsa (which is deprecated) means that Libssh2`s CNG backend currently isn’t super useful. I hope that’ll change – but for now, it means that IAP Desktop will continue to use the OpenSSL backend.

• Share this article: