SSL handshake alert: unrecognized_name error since upgrade to Java 1.7.0

Question

I upgraded from Java 1.6 to Java 1.7 today. Since then an error occur when I try to establish a connection to my webserver over SSL:

javax.net.ssl.SSLProtoco         


        

Answer 1

Java 7 introduced SNI support which is enabled by default. I have found out that certain misconfigured servers send an "Unrecognized Name" warning in the SSL handshake which is ignored by most clients... except for Java. As @Bob Kerns mentioned, the Oracle engineers refuse to "fix" this bug/feature.

As workaround, they suggest to set the jsse.enableSNIExtension property. To allow your programs to work without re-compiling, run your app as:

java -Djsse.enableSNIExtension=false yourClass

The property can also be set in the Java code, but it must be set before any SSL actions. Once the SSL library has loaded, you can change the property, but it won't have any effect on the SNI status. To disable SNI on runtime (with the aforementioned limitations), use:

System.setProperty("jsse.enableSNIExtension", "false");

The disadvantage of setting this flag is that SNI is disabled everywhere in the application. In order to make use of SNI and still support misconfigured servers:

1. Create a SSLSocket with the host name you want to connect to. Let's name this sslsock.
2. Try to run sslsock.startHandshake(). This will block until it is done or throw an exception on error. Whenever an error occurred in startHandshake(), get the exception message. If it equals to handshake alert: unrecognized_name, then you have found a misconfigured server.
3. When you have received the unrecognized_name warning (fatal in Java), retry opening a SSLSocket, but this time without a host name. This effectively disables SNI (after all, the SNI extension is about adding a host name to the ClientHello message).

For the Webscarab SSL proxy, this commit implements the fall-back setup.

Answer 2

Instead of relying on the default virtual host mechanism in apache, you can define one last catchall virtualhost that uses an arbitrary ServerName and a wildcard ServerAlias, e.g.

ServerName catchall.mydomain.com
ServerAlias *.mydomain.com

In that way you can use SNI and apache will not send back the SSL warning.

Of course, this only works if you can describe all of your domains easily using a wildcard syntax.

Answer 3

I had the same problem with an Ubuntu Linux server running subversion when accessed via Eclipse.

It has shown that the problem had to do with a warning when Apache (re)started:

[Mon Jun 30 22:27:10 2014] [warn] NameVirtualHost *:80 has no VirtualHosts

... waiting [Mon Jun 30 22:27:11 2014] [warn] NameVirtualHost *:80 has no VirtualHosts

This has been due to a new entry in ports.conf, where another NameVirtualHost directive was entered alongside the directive in sites-enabled/000-default.

After removing the directive in ports.conf, the problem had vanished (after restarting Apache, naturally)

Answer 4

Use:

• System.setProperty("jsse.enableSNIExtension", "false");
• Restart your Tomcat (important)

Answer 5

You can disable sending SNI records with the System property jsse.enableSNIExtension=false.

If you can change the code it helps to use SSLCocketFactory#createSocket() (with no host parameter or with a connected socket). In this case it will not send a server_name indication.