problem with fetching secure pages 2

[developerinlondon]

I am getting the following error trying to fetch a secure url:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

This is happening because the remote site is using a test/dummy certificate which is not recognized. But I need to find a way to ignore it.
From google I got the impression this can be fixed by configuring using keytool and adding a certificate for it. I went through the following document -

http://www.xmlmind.com/xmleditor/changes.html

which explains how to add, but where do I get the .pem file from?

Any help would be appreciated!

thanks

 

[developerinlondon]

OK. I finally found a solution to this. We can get java to not use the default checking by creating our own securitymanager and hostnameverifier. The code can look something like the following -

URLConnection conn = (URLConnection) url.openConnection();
conn = null;
if (this.url.startsWith("https")) {
conn = (HttpsURLConnection) url.openConnection();
    logger.info("Using custom security manager for secure connection.");
} else {
conn = (URLConnection) url.openConnection();
}

/***********************************************************************************
 * create a dummy security manager that will skip checking hostnames/certificates! *
 ***********************************************************************************/
if (this.url.startsWith("https")) {
    try {
        HttpsURLConnection conn_secure = (HttpsURLConnection) url.openConnection();
        SSLContext sc = SSLContext.getInstance("SSL");

        // Create empty HostnameVerifier
        HostnameVerifier hv = new HostnameVerifier() {
            public boolean verify(String urlHostName,SSLSession session) {
        logger.info("Warning: URL Host: "+urlHostName+" vs."+session.getPeerHost());
        return true;
            }
        };
        // Create a trust manager that does not validate certificate chains
        TrustManager[] trustAllCerts = new TrustManager[]{
            new X509TrustManager() {
                public java.security.cert.X509Certificate[]
getAcceptedIssuers() {
                    return null;
                }
                public void checkClientTrusted(
                    java.security.cert.X509Certificate[] certs,
String authType) {
                }
                public void checkServerTrusted(
                    java.security.cert.X509Certificate[] certs,
String authType) {
                }
            }
        };
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        SSLSocketFactory sslSocketFactory = sc.getSocketFactory();

        conn_secure.setDefaultSSLSocketFactory(sslSocketFactory);
        conn_secure.setDefaultHostnameVerifier(hv);
        
        conn = conn_secure;
    } catch (Exception e) {
          logger.error("Could not create SSLSocketFactory => " + e);
          throw new RuntimeException(e);
    }
}
// end of trust anyone code!

 

[developerinlondon]

sorry, securitymanager is meant to be trustmanager in the comments.

 

[sedj]

Thanks for posting the solution, it will hopefully help someone else !

--------------------------------------------------
Free Database Connection Pooling Software

http://www.primrose.org.uk

 

[developerinlondon]

ok suddenly for a certain operator our above script stopped working. it dies while trying to read... it works for other operators and it works from another machine with the same script. any ideas what might be wrong and where i should look at? only difference in the machine that works is we dont get any log reports of hostnameverifier class being called...

 

[developerinlondon]

later on in the program i have the following code:

            /***********************************
            * read the reply and put in xml_in *
            ************************************/
            logger.info("Reading Reply from "+this.url+".");
            // Get the response
            DataInputStream rd = new DataInputStream(conn.getInputStream());
            logger.debug("read input");
            byte[] buffer = new byte[4096];
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            logger.debug("created output buffer!");
            int i = -1;
            while (true) {
                i = rd.read(buffer, 0, buffer.length);
                if (i == -1) break;
                baos.write(buffer, 0, i);
            }

the page seems to throw an IO exception after the first log entry. ie on line commencing ataInputStream rd = new

What I dont understand is why it works for others and it works on a different machine... and it was working fine on this machine too...

 

[developerinlondon]

OK. Finally got to fix it and also learnt the original solution I put above can be written a bit more neatly like the code below:

           /***********************************************************************************
           * create a dummy security manager that will skip checking hostnames/certificates! *
           ***********************************************************************************/
              try {
                  SSLContext sc = SSLContext.getInstance("SSL");

                  // Create empty HostnameVerifier
                  HostnameVerifier hv = new HostnameVerifier() {
                      public boolean verify(String urlHostName,SSLSession session) {
                  logger.info("Warning: URL Host: "+urlHostName+" vs."+session.getPeerHost());
                  return true;
                      }
                  };
                  // Create a trust manager that does not validate certificate chains
                  TrustManager[] trustAllCerts = new TrustManager[]{
                      new X509TrustManager() {
                          public java.security.cert.X509Certificate[]
  getAcceptedIssuers() {
                              return null;
                          }
                          public void checkClientTrusted(
                              java.security.cert.X509Certificate[] certs,
  String authType) {
                          }
                          public void checkServerTrusted(
                              java.security.cert.X509Certificate[] certs,
  String authType) {
                          }
                      }
                  };
                  sc.init(null, trustAllCerts, new java.security.SecureRandom());
                  SSLSocketFactory sslSocketFactory = sc.getSocketFactory();

                  HttpsURLConnection.setDefaultSSLSocketFactory(sslSocketFactory);
                  HttpsURLConnection.setDefaultHostnameVerifier(hv);

              } catch (Exception e) {
                    logger.error("Could not create SSLSocketFactory => " + e);
                    throw new RuntimeException(e);
              }
          // end of trust anyone code!

few points to note here:
1) any URLConnection class that gets instantiated with a 'https' URL *automatically* gets converted to HttpsURLConnection type.
2) Setting the default socketfactory and HostnameVerifier for HttpsURLConnection is sufficient to put our dummy security manager in place.
3) the dummy security manager only adds the ability to deal with certificates from untrusted sources but it still checks the certificates if they are valid or not.
4) making use of the debug mode by doing "java -Djavax.net.debug=all filename > out.txt" is probably the easiest way to go about troubleshooting these types of problems as it gives a detailed output of exactly what is going on.

it so turned out the source I was getting the data from had given me a wrong certificate which was causing everything to fail.

hopefully this modified version and points will save someone else the loads of hours in fixing this!

ciao.

 

[schnirkel]

Thanks developerinlondon, your solution really saved my day !

ciao,
schnirkel from vienna