SSL Handshake fails after clienthello

Question

EDIT: I\'ll leave this as a good example of debugging SSL.

Final analysis: We had a networking issue in which one of our routers was misconfigured for a totally dif

Answer 1

For folks who come across this issue, I created a checklist:

• Make sure all TLS versions are enabled in Internet Explorer (This is for testing. You can later disable the unsecure versions once you find out the root cause)

• Check the registry keys below to make sure what you set in Internet Explorer is applied at registry level. If there are working and non-working servers, mirror the working server’s settings HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols

• Collect a network trace from the client side. Check if the client and server are agreeing on a cipher suite. If they are not, make sure the client uses the cipher suites the server is trying to use. The Group Policy setting is below Computer Configuration > Administrative Templates > Network > SSL> Configuration Settings > SSL Cipher Suite Order

• If the issue sill exists, look for any network device (proxy, firewall, load balancer etc.) in between that might be blocking TLS traffic

• Check the website binding in IIS. Make sure the certificate is valid and the port is set to 443

• Make sure the port 443 is listened in the server (netstat -an -p TCP | find /I "listening"). More details: Port 80 and 443 are not listened in IIS server Change the port number to 444 and test website. If it is accessible, it means there is a software blocking or overwriting 443 port. More details

• Disable Windows Firewall (If it works, you can enable it back and set up rules accordingly)

• Look for any third-party applications such as antivirus or endpoint protection software in the server such as Symantec Endpoint Security and Symantec Data Center Security Server Agent (Based on this document, Security Server Agent uses port 443). Uninstall them (Don’t just disable. Uninstall completely. If it works, you can install them back and configure them accordingly)

• Check if there is any Microsoft software that is using port 443. Applications like SQL Server Reporting Services (SSRS) and Windows Admin Center might interfere the port 443. An example

Source: The missing Server Hello in TLS handshake (ERR_SSL_PROTOCOL_ERROR