Nginx upstream key exchange issue

Continue with my previous post.

Now it is about phase #3. Theoretically, it should be very straightward. All Nginx needs to do is just fowarding the Akamai reqeusts to the upstream. Right? But in real life, whatever can go wrong will go wrong 😦

In the testing, we always get ‘502 Bad Gateway‘ error. I enabled debug mode in Nginx and found the relevant message ‘upstream prematurely closed connection while reading response header from upstream‘.

At the first begining I thought it could be caused by the whitelisting. But confirmed with the upstream vendor, there is no whitelisting and all settings look good to them. On our end, all settings look good to me too.

I use tcpdump to capture the conversation between Nginx and the upstream server. All I can see is that the upstream server terminates the connection after 20 seconds. We use Nginx default proxy timeout settings which is 60 seconds. So it should not be the cause. For some reason, the upstream just refuses Nginx’s request. As it is https, the first step should be ssl handshake. With this thought in mind, I scanned the upstream’s cert with ssl labs. And it does not look nice at all.

Not sure if it is the cause, but it is worthwhile to ask them to fix it anyway. Then the vendor did some fixes, and second scan looks better.

Test again, and it works!! I don’t know exactly what the magic is. But I do notice the major difference is about the ‘Key Exchange‘ of the cert. I assume what happened is Nginx requires a securer SSL handshake than the previous cert can support, so it just terminates the connection.