56

I have a x.example which serves traffic for both a.example and b.example. x.example has certificates for both a.example and b.example. The DNS for a.example and b.example is not yet set up.

If I add an /etc/hosts entry for a.example pointing to x.example's ip and run curl -XGET https://a.example, I get a 200.

However if I run curl --header 'Host: a.example' https://x.example, I get:

curl: (51) SSL: no alternative certificate subject name matches target host name x.example

I would think it would use a.example as the host. Maybe I'm not understanding how SNI/TLS works.

Because a.example is an HTTP header the TLS handshake doesn't have access to it yet? But the URL itself it does have access to?

1

3 Answers 3

74

Indeed SNI in TLS does not work like that. SNI, as everything related to TLS, happens before any kind of HTTP traffic, hence the Host header is not taken into account at that step (but will be useful later on for the webserver to know which host you are connecting too).

So to enable SNI you need a specific switch in your HTTP client to tell it to send the appropriate TLS extension during the handshake with the hostname value you need.

In case of curl, you need at least version 7.18.1 (based on https://curl.haxx.se/changes.html) and then it seems to automatically use the value provided in the Host header. It alo depends on which OpenSSL (or equivalent library on your platform) version it is linked to.

See point 1.10 of https://curl.haxx.se/docs/knownbugs.html that speaks about a bug but explains what happens:

When given a URL with a trailing dot for the host name part: "https://example.com./", libcurl will strip off the dot and use the name without a dot internally and send it dot-less in HTTP Host: headers and in the TLS SNI field.

The --connect-to option could also be useful in your case. Or --resolve as a substitute to /etc/hosts, see https://curl.haxx.se/mail/archive-2015-01/0042.html for am example, or https://makandracards.com/makandra/1613-make-an-http-request-to-a-machine-but-fake-the-hostname You can add --verbose in all cases to see in more details what is happening. See this example: https://www.claudiokuenzler.com/blog/693/curious-case-of-curl-ssl-tls-sni-http-host-header ; you will also see there how to test directly with openssl.

If you have a.example in your /etc/hosts you should just run curl with https://a.example/ and it should take care of the Host header and hence SNI (or use --resolve instead)

So to answer your question directly, replace

curl --header 'Host: a.example' https://x.example

with

curl --connect-to a.example:443:x.example:443 https://a.example

and it should work perfectly.

2
  • 1
    In curl version 7.58.0, it doesn't, and the manpage says it doesn't: It does NOT affect the hostname/port that is used for TLS/SSL (e.g. SNI, certificate verification). Commented Jun 17, 2022 at 8:29
  • For my use case the --connect-to parameter was perfect. Thank you!
    – Boldewyn
    Commented Feb 27 at 8:22
35

The selected answer helped me find the answer, even though it does not contain the answer. The answer in the mail/archive link Patrick Mevzek provided has the wrong port number. So even following that answer will cause it to continue to fail.

I used this container to run a debugging server to inspect the requests. I highly suggest anyone debugging this kind of issue do the same.

Here is how to address the OP's question.

# Instead of this:
# curl --header 'Host: a.example'        https://x.example

# Do:
  host=a.example
  target=x.example

  ip=$(dig +short $target | head -n1)
  curl -sv   --resolve $host:443:$ip https://$host

If you want to ignore bad certificates matches, use -svk instead of -sv

curl -svk --resolve $host:443:$ip https://$host

Note: Since you are using https, you must use 443 in the --resolve argument instead of 80 as was stated on the mail/archive

6
  • 1
    Very useful on a Mac as Catalina no longer seems to let you manually configure your hosts file. Commented Feb 4, 2020 at 18:13
  • 1
    instead ` ip=$(dig +short x.example | head -n1)` use ` ip=$(dig +short "$target" | head -n1)`
    – Rax
    Commented Nov 2, 2020 at 14:17
  • Doh! Corrected. Thank you, @Rax. (I miss your roast beef sandwiches. Arby's is inferior.) Commented Dec 3, 2020 at 19:53
  • 1
    @SamCritchley I don't think that's correct. /etc/hosts still works, but you need the right context. And it isn't the OS that should support it or not, but the apps/browser. Safari, for example, sends a https binding dns request and if the gets an answer, it ignores /etc/hosts.
    – Lethargos
    Commented Oct 8, 2021 at 8:24
  • Yeah, you're right @Lethargos, sorry about that. Commented Oct 25, 2021 at 13:41
7

I had a similar need. Didn't have sudo access to update hosts file.

I use resolve parameter and also added the DNS host name as a header parameter.

--resolve <dns name>:<port>:<ip addr>

curl --request POST --resolve dns_name:443:a.b.c.d 'https://dns_name/x/y' --header 'Host: dns_name' ....

Cheers..

Not the answer you're looking for? Browse other questions tagged or ask your own question.