.NET Core 3.0 Preview 8 "SSL Handshake failed with OpenSSL error" when running via linux docker container

[f2calv]

SSL Handshake failed with OpenSSL error

I'm upgrading .NET Core 2.2 kubernetes-hosted apps to .NET Core 3.0 preview 8, these apps connect to external SSL-secured APIs.

The upgraded apps run as expected on Windows development machine, and the .NET Core 2.2 docker image also works fine - but the .NET Core 3.0 preview 8 docker image version fails with;

System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.
---> Interop+Crypto+OpenSslCryptographicException: error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small
--- End of inner exception stack trace ---
at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 recvCount, Byte[]& sendBuf, Int32& sendCount)
at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteContext& context, ArraySegment1 inputBuffer, Byte[]& outputBuffer, SslAuthenticationOptions sslAuthenticationOptions) --- End of inner exception stack trace --- at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest) --- End of stack trace from previous location where exception was thrown --- at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at SharedLib.Class1.TryConnect() in /src/SharedLib/Class1.cs:line 17

The above issue is similar, but not identical to #https://github.com/dotnet/corefx/issues/37927

General

I have created a repo to quickly reproduce the issue, see here.

Test steps, open BugCheck.sln with Visual Studio 2019 16.3 Preview 2 on Windows 10;

• Run ConsoleApp22 - HttpClient works as expected
• Run ConsoleApp30 - HttpClient works as expected

Now use Docker Desktop 2.1.0.1 for Windows 10;

• Build .NET Core 2.2 image by running ./docker-build-ConsoleApp22.bat
  -- Run .NET Core 2.2 container by running ./docker-run-ConsoleApp22.bat <-- HttpClient works as expected
• Build .NET Core 3.0 image by running ./docker-build-ConsoleApp30.bat
  -- Run .NET Core 3.0 container by running ./docker-run-ConsoleApp30.bat <-- HttpClient fails with the above OpenSSL error.

The test URL I'm connecting to is this, maybe their certificate is poor quality and/or .NET Core 3 under linux requires higher levels of SSL security? Either way it would be good to know the exact cause...

[bartonjs]

System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.
---> Interop+Crypto+OpenSslCryptographicException: error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small

error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small

According to https://www.ssllabs.com/ssltest/analyze.html?d=api-fxpractice.oanda.com their key exchanges are preferring DHE-1024 over ECDHE. Using the guidance from NIST SP 800-57, a 1024-bit DHE key has 80 bits of security (or smaller).

Debian Buster has raised the OpenSSL TLS security level to 2 (https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#openssl-defaults), which requires DHE at 2048-bit or higher (112 bits of security).

Theoretically, editing /etc/ssl/openssl.cnf and setting CipherString = DEFAULT:@SECLEVEL=1 will change the security level back to 1.

[bartonjs]

Other options:

• Change from Debian to a less opinionated-about-OpenSSL-configuration distro
• Set the environment variable CLR_OPENSSL_VERSION_OVERRIDE to 1.0.0, 1.0.2, or 1.0 ... depending on which one Debian Buster uses for OpenSSL 1.0.x's SONAME (locate openssl.so.1.0, use the value for whatever comes after .so. in the answer)
  • .NET Core 3.0 changed to prefer 1.1 over 1.0, .NET Core 2.x prefers 1.0 over 1.1 (because it used to be 1.0-only)

[f2calv]

@bartonjs thanks for the fast and detailed response to this :)
Your first suggestion, to change the CipherString back to SECLEVEL=1 fixed the issue and was easiest to accomplish ...especially for a linux noob like me!

Below is the exact line I added to my Dockerfile;

RUN sed 's/DEFAULT@SECLEVEL=2/DEFAULT@SECLEVEL=1/' /etc/ssl/openssl.cnf > /etc/ssl/openssl.cnf.changed && mv /etc/ssl/openssl.cnf.changed /etc/ssl/openssl.cnf

I have updated my demonstration repo accordingly.

[dima-hx]

System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.
---> Interop+Crypto+OpenSslCryptographicException: error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small

error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small

According to https://www.ssllabs.com/ssltest/analyze.html?d=api-fxpractice.oanda.com their key exchanges are preferring DHE-1024 over ECDHE. Using the guidance from NIST SP 800-57, a 1024-bit DHE key has 80 bits of security (or smaller).

Debian Buster has raised the OpenSSL TLS security level to 2 (https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#openssl-defaults), which requires DHE at 2048-bit or higher (112 bits of security).

Theoretically, editing /etc/ssl/openssl.cnf and setting CipherString = DEFAULT:@SECLEVEL=1 will change the security level back to 1.

It is just a matter of editing file /etc/ssl/openssl.cnf changing last line
from:
CipherString = DEFAULT@SECLEVEL=2
to
CipherString = DEFAULT@SECLEVEL=1

Or add to DockerFile

RUN sed -i "s|DEFAULT@SECLEVEL=2|DEFAULT@SECLEVEL=1|g" /etc/ssl/openssl.cnf

[thequy]

Following your instructions, I did something and it worked:

• Copy openssl.cnf file from running container to host:

Eg:

docker cp dotnet:/etc/ssl/openssl.cnf /share/ssl

• Edit file content (as some comments above):

Eg:

[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=1 << this line 💯

• Add mapping to docker run command:

Eg:

docker run --rm --name poc -p 80:80 -p 443:443 -v /share/dotnet/poc:/app -v /share/dotnet/https:/https -v /share/ssl/openssl.cnf:/etc/ssl/openssl.cnf -w /app -e ASPNETCORE_URLS="https://+;http://+" -e ASPNETCORE_HTTPS_PORT=443 -e ASPNETCORE_Kestrel__Certificates__Default__Password="123456" -e ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx mcr.microsoft.com/dotnet/core/aspnet dotnet PoC.dll

Thanks! 👍