Convince nginx to terminate the connection if none of my virtual host matches the server name sent by the client?

Question

I'm painfully aware that using the ssl equivalent of:

server {
        listen 80;
        return 444;
}

kills all ssl exchanges, meanwhile, without something of the sort, nginx happily serve a random virtual host. Instead of trying to gracefully end the connection, I would like nginx to simply terminate the tcp connection. If I'm not mistaken, the ssl has to send the domain as part of the negotiation. I would like nginx to simply close the connection if it can't find a virtual host that matches the domain the client is asking for. How could I do this? Should I use a middleman?

If a correct domain is give:

1. C: (TLS Handshake) Hello, I support XYZ Encryption, and I am trying to connect to 'site.example.com'.
2. S: (TLS Handshake) Hi There, Here is my Public Certificate, and lets use this encryption algorithm.

Now an invalid one:

1. C: (TLS Handshake) Hello, I support XYZ Encryption, and I am trying to connect to 'invalid-domain.com'.
2. S: CLOSE