X509Chain.Build() returns true for partial chain when AllowUnknownCertificateAuthority flag is present

[stewartadam]

Description

According this comment by @bartonjs in #26449 about the AllowUnknownCertificateAuthority flag (emphasis mine):

If AllowUnknownCertificateAuthority is the only flag set then chain.Build() will return true if

• The chain correctly terminated in a self-signed certificate (via ExtraStore, or searched persisted stores)
• ...

So, Build() returns true, you know that a time-valid non-revoked chain is present.

1. the docs make no mention of it being required to be the only flag set, so if this is accurate docs need updated to reflect this.
2. I cannot reproduce the above intended behavior, and instead see X509Chain.Build() returning true for any leaf certificate and any CA added to the ExtraStore combination, even the leaf certificate is not signed by the ExtraStore's CA.

Configuration

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.201
 Commit:    a09bd5c86c

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.0
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.201/

Host (useful for support):
  Version: 5.0.4
  Commit:  f27d337295

.NET SDKs installed:
  3.1.302 [/usr/local/share/dotnet/sdk]
  5.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Repro

Run in bash:

create_ca() {
  local CA_CN="$1"
  openssl genrsa -out ${CA_CN}.key.pem 2048 # Generate private key
  openssl req -x509 -new -nodes -key ${CA_CN}.key.pem -sha256 -days 1825 -out ${CA_CN}.cert.pem -subj "/CN=${CA_CN}/O=MyDevices/C=US" # Generate root certificate
}

create_leaf_cert() {
  local DEVICE_CN="$1"
  openssl genrsa -out ${DEVICE_CN}.key.pem 2048 # new private key
  openssl req -new -key ${DEVICE_CN}.key.pem -out ${DEVICE_CN}.csr.pem -subj "/CN=${DEVICE_CN}/O=MyDevices/C=US" # generate signing request for the CA
}

sign_leaf_cert() {
  local DEVICE_CN="$1"
  local CA_CN="$2"
  openssl x509 -req -in ${DEVICE_CN}.csr.pem -CA "${CA_CN}.cert.pem" -CAkey "${CA_CN}.key.pem" -set_serial 01 -out "${DEVICE_CN}.cert.pem" -days 825 -sha256 # sign the CSR
}

# Create one self-issued CA + signed cert
create_ca trust.mydevices.com
create_leaf_cert dev01.mydevices.com
sign_leaf_cert dev01.mydevices.com trust.mydevices.com

# Create another self-issued CA + signed cert
create_ca trust.different.com
create_leaf_cert dev01.different.com
sign_leaf_cert dev01.different.com trust.different.com

Expected behavior

I expect chain.Build() to return the boolean equivalent to the OpenSSL validations when paired with the various certificates:

openssl verify -CAfile trust.mydevices.com.cert.pem dev01.mydevices.com.cert.pem # true
openssl verify -CAfile trust.mydevices.com.cert.pem dev01.different.com.cert.pem # false - mismatched CA

openssl verify -CAfile trust.different.com.cert.pem dev01.different.com.cert.pem # true
openssl verify -CAfile trust.different.com.cert.pem dev01.mydevices.com.cert.pem # false - mismatched CA

Observed behavior

success=true, even when PartialChain is present in the status because the chain could not be verified

$ dotnet script main.csx -- trust.mydevices.com.cert.pem dev01.mydevices.com.cert.pem # true
Success: True
Chain Status: flag=UntrustedRoot info=The certificate was not trusted.

$ dotnet script main.csx -- trust.mydevices.com.cert.pem dev01.different.com.cert.pem # true - despite mismatched CA
Success: True
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

$ dotnet script main.csx -- trust.different.com.cert.pem dev01.different.com.cert.pem # true
Success: True
Chain Status: flag=UntrustedRoot info=The certificate was not trusted.

$ dotnet script main.csx -- trust.different.com.cert.pem dev01.mydevices.com.cert.pem # true - despite mismatched CA
Success: True
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

According this comment by @bartonjs about the AllowUnknownCertificateAuthority flag (emphasis mine):

If AllowUnknownCertificateAuthority is the only flag set then chain.Build() will return true if

• The chain correctly terminated in a self-signed certificate (via ExtraStore, or searched persisted stores)
• ...

So, Build() returns true, you know that a time-valid non-revoked chain is present.

1. the docs make no mention of it being required to be the only flag set, so if this is accurate docs need updated to reflect this.
2. I cannot reproduce the above intended behavior, and instead see X509Chain.Build() returning true for any leaf certificate and any CA added to the ExtraStore combination, even the leaf certificate is not signed by the ExtraStore's CA.

Configuration

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.201
 Commit:    a09bd5c86c

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.0
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.201/

Host (useful for support):
  Version: 5.0.4
  Commit:  f27d337295

.NET SDKs installed:
  3.1.302 [/usr/local/share/dotnet/sdk]
  5.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Repro

Run in bash:

create_ca() {
  local CA_CN="$1"
  openssl genrsa -out ${CA_CN}.key.pem 2048 # Generate private key
  openssl req -x509 -new -nodes -key ${CA_CN}.key.pem -sha256 -days 1825 -out ${CA_CN}.cert.pem -subj "/CN=${CA_CN}/O=MyDevices/C=US" # Generate root certificate
}

create_leaf_cert() {
  local DEVICE_CN="$1"
  openssl genrsa -out ${DEVICE_CN}.key.pem 2048 # new private key
  openssl req -new -key ${DEVICE_CN}.key.pem -out ${DEVICE_CN}.csr.pem -subj "/CN=${DEVICE_CN}/O=MyDevices/C=US" # generate signing request for the CA
}

sign_leaf_cert() {
  local DEVICE_CN="$1"
  local CA_CN="$2"
  openssl x509 -req -in ${DEVICE_CN}.csr.pem -CA "${CA_CN}.cert.pem" -CAkey "${CA_CN}.key.pem" -set_serial 01 -out "${DEVICE_CN}.cert.pem" -days 825 -sha256 # sign the CSR
}

# Create one self-issued CA + signed cert
create_ca trust.mydevices.com
create_leaf_cert dev01.mydevices.com
sign_leaf_cert dev01.mydevices.com trust.mydevices.com

# Create another self-issued CA + signed cert
create_ca trust.different.com
create_leaf_cert dev01.different.com
sign_leaf_cert dev01.different.com trust.different.com

Expected behavior

I expect chain.Build() to return the boolean equivalent to the OpenSSL validations when paired with the various certificates:

openssl verify -CAfile trust.mydevices.com.cert.pem dev01.mydevices.com.cert.pem # true
openssl verify -CAfile trust.mydevices.com.cert.pem dev01.different.com.cert.pem # false - mismatched CA

openssl verify -CAfile trust.different.com.cert.pem dev01.different.com.cert.pem # true
openssl verify -CAfile trust.different.com.cert.pem dev01.mydevices.com.cert.pem # false - mismatched CA

Observed behavior

success=true, even when PartialChain is present in the status because the chain could not be verified

$ dotnet script main.csx -- trust.mydevices.com.cert.pem dev01.mydevices.com.cert.pem # true
Success: True
Chain Status: flag=UntrustedRoot info=The certificate was not trusted.

$ dotnet script main.csx -- trust.mydevices.com.cert.pem dev01.different.com.cert.pem # true - despite mismatched CA
Success: True
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

$ dotnet script main.csx -- trust.different.com.cert.pem dev01.different.com.cert.pem # true
Success: True
Chain Status: flag=UntrustedRoot info=The certificate was not trusted.

$ dotnet script main.csx -- trust.different.com.cert.pem dev01.mydevices.com.cert.pem # true - despite mismatched CA
Success: True
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

Author:	stewartadam
Assignees:	-
Labels:	area-System.Security, untriaged
Milestone:	-

[stewartadam]

Side note, in .NET 5.0 support for certificate pinning was added with CustomRootStore and that mode works as expected:

...
X509Chain chain = new X509Chain();
chain.ChainPolicy.CustomTrustStore.Add(caCert);
chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;
chain.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
var success = chain.Build(leafCert);
...

This, without the AllowUnknownCertificateFlag, behaves as we expected:

$ dotnet script main-customstore.csx -- trust.mydevices.com.cert.pem dev01.mydevices.com.cert.pem # true
Success: True
Chain Status: N/A (no flags)

$ dotnet script main-customstore.csx -- trust.mydevices.com.cert.pem dev01.different.com.cert.pem # false - mismatched CA
Success: False
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

$ dotnet script main-customstore.csx -- trust.different.com.cert.pem dev01.different.com.cert.pem # true
Success: True
Chain Status: N/A (no flags)

$ dotnet script main-customstore.csx -- trust.different.com.cert.pem dev01.mydevices.com.cert.pem # false - mismatched CA
Success: False
Chain Status: flag=PartialChain info=One or more certificates required to validate this certificate cannot be found.

[bartonjs]

the docs make no mention of it being required to be the only flag set, so if this is accurate docs need updated to reflect this.

The "only" is just because once multiple flags are set the statement of what returns true and what returns false becomes complicated. Ease of scope for discussion, not a technical limitation.

The chain correctly terminated in a self-signed certificate (via ExtraStore, or searched persisted stores)

Yeah, apparently that was a goof on my part. It's the way I always expect it to work, and I forget that PartialChain is also suppressed by AllowUnknownCertificateAuthority.

On Windows, that behavior is controlled by Windows. On non-Windows we maintain parity with what Windows does since that describes the ".NET Framework behavior" for this type, which is the strong starting point for the ".NET behavior" for this type.

[stewartadam]

@bartonjs thanks for the clarification. May I request this be re-opened and track a docs change for these classes then? At the present time, AllowUnknownCertificateAuthority is described as Ignore that the chain cannot be verified due to an unknown certificate authority (CA). here which is misleading because it doesn't just suppress UntrustedRoot, but PartialChain as well.

I understand the reason behind keeping the behavior consistent between platforms and therefore why it does so, but that's definitely not intuitive behavior nor described by the docs - potentially with disastrous consequences because if users are unware of this, AllowUnknownCertificateAuthority will allow any two unrelated certs to pass validation.

Given the present behavior, IMO it's crucially important the docs reference the type of verification you describe in the linked GitHub issue from the original post, requiring a manual validation of chain.ChainElements[chain.ChainElements.Count - 1].Certificate when using AllowUnknownCertificateAuthority (or else using the new .NET 5-based cert pinning APIs).