Using libssh2 with CryptoNG and ECDSA
Libssh2 lets us choose between multiple different crypto backends. On Windows, an attractive
option is to use the WinCNG backend. CNG is part of the operating system,
it’s maintained by Microsoft and serviced by Windows update, and reusing OS functionality helps us keep our binaries small.
However, the WinCNG backend used to have a significant limitation: it lacked support for ECDSA.
Given the widespread use of ECDSA for host and user authentication, this limitation often made OpenSSL
a more attractive choice, even on Windows systems.
Not being a huge fan of OpenSSL, I decided to do something about that, got my hands dirty, and
added ECDSA support to libssh2’s CNG backend.
The feature is already available in master and should be part of the upcoming 1.12 release.
Building libssh2 with WinCNG and ECDSA support
To build libssh2 with WinCNG and ECDSA support, we have to use two build options:
CRYPTO_BACKEND=WinCNG: This (existing) option enables the WinCNG backend.ENABLE_ECDSA_WINCNG=ON: This new option enables ECDSA support in the WinCNG backend.
The implementation uses a feature of BCryptDeriveKey that’s only available in Windows 10 and later versions of Windows.
Any attempt in using ECDSA on older Windows versions is
bound to fail in a Unable to exchange encryption keys error. It’s for that reason that
ECDSA support needs to be enabled explicitly using the ENABLE_ECDSA_WINCNG option.
Building with cmake
Freed of the OpenSSL dependency, it’s now rather straightforward to build libssh2 without vcpkg.
The following example makefile, taken from the IAP Desktop source code
shows how we can build libssh2.dll with ECDSA support, statically-linked C runtime, and debug symbols:
LIBSSH2_OPTIONS = \
-DBUILD_STATIC_LIBS=OFF \
-DBUILD_SHARED_LIBS=ON \
-DCRYPTO_BACKEND=WinCNG \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTING=OFF \
-DENABLE_ZLIB_COMPRESSION=OFF \
-DENABLE_ECDSA_WINCNG=ON \
-DENABLE_DEBUG_LOGGING=ON \
-DCMAKE_POLICY_DEFAULT_CMP0091=NEW
!if ("$(PLATFORM)" == "x86")
CMAKE_PLATFORM = Win32
!else
CMAKE_PLATFORM = $(PLATFORM)
!endif
!if ("$(CONFIGURATION)" == "Debug")
CMAKE_CONFIGURATION = Debug
LIBSSH2_OPTIONS = $(LIBSSH2_OPTIONS) -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDebug
!else
CMAKE_CONFIGURATION = RelWithDebInfo
LIBSSH2_OPTIONS = $(LIBSSH2_OPTIONS) -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded
!endif
CMAKE = cmake
CMAKE_GENERATOR = "Visual Studio 17 2022"
CMAKE_OPTS = -Wno-dev -DCMAKE_BUILD_TYPE=$(CONFIGURATION) -A $(CMAKE_PLATFORM)
CMAKE_BUILDDIR = build\$(PLATFORM)
build:
$(CMAKE) \
$(CMAKE_OPTS) \
-G $(CMAKE_GENERATOR) \
$(LIBSSH2_OPTIONS) \
-B $(CMAKE_BUILDDIR)
$(CMAKE) --build $(CMAKE_BUILDDIR) \
--config $(CMAKE_CONFIGURATION)