Articles / Security

826 words

  • tls
  • x509
  • openssl
  • pki
  • certificates
  • debugging

The Intermediate Is Sent, So Why Does Validation Still Fail?

You run openssl s_client -showcerts, see two certificates in the output, and the client still says "unable to get local issuer certificate". The intermediate is right there. So what is wrong?

Sending a certificate is not the same as sending the right one. The client has to build a path from your leaf to a root it trusts, and every link must be valid, not merely present. Here are the usual reasons a link breaks, in the order I would check them.

First, look at what the server really sends

Do not trust the file on disk. Ask the live server, with SNI set, because the wrong virtual host often serves a different chain.

openssl s_client -connect example.com:443 -servername example.com \
  -showcerts -verify_return_error </dev/null 2>&1 | less

The "Certificate chain" section lists each certificate with its subject (s:) and issuer (i:). Save each PEM block to its own file (leaf.pem, inter.pem) so you can inspect them one at a time.

Cause 1: the issuer name matches but the key does not

Matching on the issuer's name is only a hint. A real verifier also checks that the leaf's signature validates under the intermediate's public key. If your CA reissued the intermediate with the same name but a new key, an old copy on your server will not verify the new leaf.

openssl x509 -in leaf.pem -noout -ext authorityKeyIdentifier
openssl x509 -in inter.pem -noout -ext subjectKeyIdentifier

The leaf's Authority Key Identifier should equal the intermediate's Subject Key Identifier. If they differ, you have the wrong intermediate, even though the names look identical. Re-download the chain your CA issued with that leaf.

Cause 2: the intermediate itself is expired or not a CA

We all remember to check the leaf's expiry. Nobody checks the intermediate, and it has its own dates.

openssl x509 -in inter.pem -noout -dates \
  -ext basicConstraints,keyUsage

You want CA:TRUE in basic constraints and Certificate Sign in key usage. A certificate missing either cannot vouch for another one, and a strict verifier will refuse it. A common accident is putting a second leaf certificate in the bundle instead of the intermediate.

Cause 3: the client picks a path ending somewhere it distrusts

This one is the interesting one. A CA can have one intermediate signed by two different roots (a cross-sign), so there are two valid-looking paths. Which one the client follows depends on its trust store and on how clever its path builder is.

Quick detour, because this caused a real mess. When an old cross-signed root expired in 2021, clients on old OpenSSL 1.0.x failed against sites whose chains were otherwise fine. As I understand it, they committed to the expired path and gave up, while newer versions tried the alternative. Same server, same certificates, different verdict. Hang on, why does that matter now? Because old embedded devices and ancient base images are still out there.

The fix on the server side is to send the chain that ends in the most widely trusted root, or the shorter one, and leave out cross-signs that only help legacy clients unless you need them.

Cause 4: order, extras and the root itself

The TLS specifications say the sender's certificate comes first, and each following certificate certifies the one before it. Many libraries sort out a jumbled bundle anyway. Some do not, and a few embedded stacks simply walk the list in order.

  • Leaf first, then the intermediate(s), in issuing order.
  • No unrelated certificates in the bundle.
  • No root. The client already has it or does not trust it; sending it adds bytes and nothing else.

If you concatenated files by hand, check the order. A bundle that starts with the intermediate is a classic.

Cause 5: the problem is on the client side

Sometimes the chain is perfect and the client's trust store is the problem. Slim container images often ship without ca-certificates. Private CAs need their root installed. And some verifiers enforce constraints the browser ignores, such as name constraints on an internal intermediate or an extended key usage that excludes server authentication.

Test that separately, with the chain supplied by you rather than the server:

openssl verify -show_chain -untrusted inter.pem leaf.pem

With -untrusted, the intermediate is treated as a helper and the system roots do the trusting. If this succeeds locally but the client fails, point at that client's trust store. If it fails here, the error code names the culprit: "certificate has expired", "unable to get issuer certificate" and "invalid CA certificate" each map neatly to one of the causes above.

A routine that finds it quickly

  1. Pull the chain from the live host with the correct SNI.
  2. Compare AKI on each certificate with SKI on the next.
  3. Check dates and CA flags on every certificate, not only the leaf.
  4. Run openssl verify -show_chain and read the exact error.
  5. Only then blame the client.

Most of the time the answer is a stale intermediate left in a bundle from the previous renewal. Serve the full chain your CA gave you with the current leaf, leaf first, no root, and the mystery goes away.

← All articles