Java and HTTPS url connection without downloading certificate

Question

This code connects to a HTTPS site and I am assuming I am not verifying the certificate. But why don\'t I have to install a certificate locally for the site? Shouldn\'t I

Answer 1

The reason why you don't have to load a certificate locally is that you've explicitly chosen not to verify the certificate, with this trust manager that trusts all certificates.

The traffic will still be encrypted, but you're opening the connection to Man-In-The-Middle attacks: you're communicating secretly with someone, you're just not sure whether it's the server you expect, or a possible attacker.

If your server certificate comes from a well-known CA, part of the default bundle of CA certificates bundled with the JRE (usually cacerts file, see JSSE Reference guide), you can just use the default trust manager, you don't have to set anything here.

If you have a specific certificate (self-signed or from your own CA), you can use the default trust manager or perhaps one initialised with a specific truststore, but you'll have to import the certificate explicitly in your trust store (after independent verification), as described in this answer. You may also be interested in this answer.

Answer 2

A simple, but not pure java solution, is to shell out to curl from java, which gives you complete control over how the request is done. If you're just doing this for something simple, this allows you to ignore certificate errors at times, by using this method. This example shows how to make a request against a secure server with a valid or invalid certificate, pass in a cookie, and get the output using curl from java.

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;

public class MyTestClass
{
  public static void main(String[] args) 
  {
    String url = "https://www.google.com";
    String sessionId = "faf419e0-45a5-47b3-96d1-8c62b2a3b558";

    // Curl options are:
    // -k: ignore certificate errors
    // -L: follow redirects
    // -s: non verbose
    // -H: add a http header

    String[] command = { "curl", "-k", "-L", "-s", "-H", "Cookie: MYSESSIONCOOKIENAME=" + sessionId + ";", "-H", "Accept:*/*", url };
    String output = executeShellCmd(command, "/tmp", true, true);
    System.out.println(output);
  }

  public String executeShellCmd(String[] command, String workingFolder, boolean wantsOutput, boolean wantsErrors)
  {
    try
    {
      ProcessBuilder pb = new ProcessBuilder(command);
      File wf = new File(workingFolder);
      pb.directory(wf);

      Process proc = pb.start();
      BufferedReader stdInput = new BufferedReader(new InputStreamReader(proc.getInputStream()));
      BufferedReader stdError = new BufferedReader(new InputStreamReader(proc.getErrorStream()));

      StringBuffer sb = new StringBuffer();
      String newLine = System.getProperty("line.separator");
      String s;

      // read stdout from the command
      if (wantsOutput)
      {
        while ((s = stdInput.readLine()) != null)
        {
          sb.append(s);
          sb.append(newLine);
        }
      }

      // read any errors from the attempted command
      if (wantsErrors)
      {
        while ((s = stdError.readLine()) != null)
        {
          sb.append(s);
          sb.append(newLine);
        }
      }

      String result = sb.toString();

      return result;
    }
    catch (IOException e)
    {
      throw new RuntimeException("Problem occurred:", e);
    }
  }


}

Answer 3

But why don't I have to install a certificate locally for the site?

Well the code that you are using is explicitly designed to accept the certificate without doing any checks whatsoever. This is not good practice ... but if that is what you want to do, then (obviously) there is no need to install a certificate that your code is explicitly ignoring.

Shouldn't I have to install a certificate locally and load it for this program or is it downloaded behind the covers?

No, and no. See above.

Is the traffic between the client to the remote site still encrypted in transmission?

Yes it is. However, the problem is that since you have told it to trust the server's certificate without doing any checks, you don't know if you are talking to the real server, or to some other site that is pretending to be the real server. Whether this is a problem depends on the circumstances.

---

If we used the browser as an example, typically a browser doesn't ask the user to explicitly install a certificate for each ssl site visited.

The browser has a set of trusted root certificates pre-installed. Most times, when you visit an "https" site, the browser can verify that the site's certificate is (ultimately, via the certificate chain) secured by one of those trusted certs. If the browser doesn't recognize the cert at the start of the chain as being a trusted cert (or if the certificates are out of date or otherwise invalid / inappropriate), then it will display a warning.

Java works the same way. The JVM's keystore has a set of trusted certificates, and the same process is used to check the certificate is secured by a trusted certificate.

Does the java https client api support some type of mechanism to download certificate information automatically?

No. Allowing applications to download certificates from random places, and install them (as trusted) in the system keystore would be a security hole.

Answer 4

Use the latest X509ExtendedTrustManager instead of X509Certificate as advised here: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints

    package javaapplication8;

    import java.io.InputStream;
    import java.net.Socket;
    import java.net.URL;
    import java.net.URLConnection;
    import java.security.cert.CertificateException;
    import java.security.cert.X509Certificate;
    import javax.net.ssl.HostnameVerifier;
    import javax.net.ssl.HttpsURLConnection;
    import javax.net.ssl.SSLContext;
    import javax.net.ssl.SSLEngine;
    import javax.net.ssl.SSLSession;
    import javax.net.ssl.TrustManager;
    import javax.net.ssl.X509ExtendedTrustManager;

    /**
     *
     * @author hoshantm
     */
    public class JavaApplication8 {

        /**
         * @param args the command line arguments
         * @throws java.lang.Exception
         */
        public static void main(String[] args) throws Exception {
            /*
             *  fix for
             *    Exception in thread "main" javax.net.ssl.SSLHandshakeException:
             *       sun.security.validator.ValidatorException:
             *           PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
             *               unable to find valid certification path to requested target
             */
            TrustManager[] trustAllCerts = new TrustManager[]{
                new X509ExtendedTrustManager() {
                    @Override
                    public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                        return null;
                    }

                    @Override
                    public void checkClientTrusted(X509Certificate[] certs, String authType) {
                    }

                    @Override
                    public void checkServerTrusted(X509Certificate[] certs, String authType) {
                    }

                    @Override
                    public void checkClientTrusted(X509Certificate[] xcs, String string, Socket socket) throws CertificateException {

                    }

                    @Override
                    public void checkServerTrusted(X509Certificate[] xcs, String string, Socket socket) throws CertificateException {

                    }

                    @Override
                    public void checkClientTrusted(X509Certificate[] xcs, String string, SSLEngine ssle) throws CertificateException {

                    }

                    @Override
                    public void checkServerTrusted(X509Certificate[] xcs, String string, SSLEngine ssle) throws CertificateException {

                    }

                }
            };

            SSLContext sc = SSLContext.getInstance("SSL");
            sc.init(null, trustAllCerts, new java.security.SecureRandom());
            HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

            // Create all-trusting host name verifier
            HostnameVerifier allHostsValid = new HostnameVerifier() {
                @Override
                public boolean verify(String hostname, SSLSession session) {
                    return true;
                }
            };
            // Install the all-trusting host verifier
            HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
            /*
             * end of the fix
             */

            URL url = new URL("https://10.52.182.224/cgi-bin/dynamic/config/panel.bmp");
            URLConnection con = url.openConnection();
            //Reader reader = new ImageStreamReader(con.getInputStream());

            InputStream is = new URL(url.toString()).openStream();

            // Whatever you may want to do next

        }

    }

Answer 5

If you are using any Payment Gateway to hit any url just to send a message, then i used a webview by following it : How can load https url without use of ssl in android webview

and make a webview in your activity with visibility gone. What you need to do : just load that webview.. like this:

 webViewForSms.setWebViewClient(new SSLTolerentWebViewClient());
                webViewForSms.loadUrl(" https://bulksms.com/" +
                        "?username=test&password=test@123&messageType=text&mobile="+
                        mobileEditText.getText().toString()+"&senderId=ATZEHC&message=Your%20OTP%20for%20A2Z%20registration%20is%20124");

Easy.

You will get this: SSLTolerentWebViewClient from this link: How can load https url without use of ssl in android webview

Answer 6

Java and HTTPS url connection without downloading certificate

If you really want to avoid downloading the server's certificate, then use an anonymous protocol like Anonymous Diffie-Hellman (ADH). The server's certificate is not sent with ADH and friends.

You select an anonymous protocol with setEnabledCipherSuites. You can see the list of cipher suites available with getEnabledCipherSuites.

Related: that's why you have to call SSL_get_peer_certificate in OpenSSL. You'll get a X509_V_OK with an anonymous protocol, and that's how you check to see if a certificate was used in the exchange.

But as Bruno and Stephed C stated, its a bad idea to avoid the checks or use an anonymous protocol.

---

Another option is to use TLS-PSK or TLS-SRP. They don't require server certificates either. (But I don't think you can use them).

The rub is, you need to be pre-provisioned in the system because TLS-PSK is Pres-shared Secret and TLS-SRP is Secure Remote Password. The authentication is mutual rather than server only.

In this case, the mutual authentication is provided by a property that both parties know the shared secret and arrive at the same premaster secret; or one (or both) does not and channel setup fails. Each party proves knowledge of the secret is the "mutual" part.

Finally, TLS-PSK or TLS-SRP don't do dumb things, like cough up the user's password like in a web app using HTTP (or over HTTPS). That's why I said each party proves knowledge of the secret...