Java applet behind proxy experiences temporary freezes when instantiating new classes

问题

Introduction

We have some problems with our Java applet at our customer's office. The applet is intended for recording the screen by taking screenshots at regular intervals. It's signed and it should run with elevated privileges.

The problems we encounter are:

The Java applet may sometimes not start at all. If the Java console shows up, it may disappear soon. It feels as if the whole JVM just crashes.
If the Java applet starts, it sometimes crashes something like 10 seconds after startup.
If the Java applet starts, it almost always experiences temporary freezes that last up to 12 seconds.
(The applet is a bit slow to start.)

A typical case is that after booting up Windows the applet won't load on the first attempt. After a refresh/re-login/restarting the browser (I'm not totally sure what's needed) the applet starts, experiences freezes and then crashes. On the third attempt it won't crash anymore but it still experiences freezes.

Background

The customer has a Blue Coat Systems HTTP proxy that probably has something to do with the problem. The network is set up so that all traffic to the Internet has to go through the proxy. For example, ping www.google.com would fail after a timeout because the host www.google.com couldn't be resolved. I have found some pages that describe problems with Java and Blue Coat proxy. I don't think these provide much helpful information but they strengthen my belief that the proxy has something to do with the problems.

Java not resolving to bluecoat proxy
Problem accessing GoToAssist, GoToMeeting, GoToWebinar, or GoToMyPC
Problem accessing Salesforce.com
I am getting a Java error when accessing RealTimeGraph with Firefox browser.

The proxy requires NTLM authentication, meaning that the first time the browser tries to go through I get a popup asking for my username and password. Here's part of the HTTP response from the proxy when I tried to access Google using Chrome:

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NTLM TlRMTVNTUAACAAAADAAMADgAAAAFgokC/+XiO/tpmQMAAAAAAAAAAKQApABEAAAABQLODgAAAA9KAEEATQBQAFQASQACAAwASgBBAE0AUABUAEkAAQASAFAASwBIAEsASQBEAEMAMAAxAAQAHgBwAHUAdQBrAGUAcwBrAHUAcwAuAGwAbwBjAGEAbAADADIAcABrAGgAawBpAGQAYwAwADEALgBwAHUAdQBrAGUAcwBrAHUAcwAuAGwAbwBjAGEAbAAFAB4AcAB1AHUAawBlAHMAawB1AHMALgBsAG8AYwBhAGwAAAAAAA==

Environment

I was at the customer's premises to experience the problem myself on my own laptop. My setup is that I run Ubuntu Linux 12.04 and Windows 7 in a virtual machine. I used Windows and Java 1.6.35 for most of the tests. I did run the applet once in Linux (using Oracle Java 1.7.0_07 I think) and it didn't experience freezes, but the screenshots that the applet took could be blank or corrupted.

I have unchecked the box "Keep temporary files on my computer" in Java settings since cached Java applets sometimes mess up with development. I had the temporary files disabled when I was debugging the applet at the customer's office, but I don't believe this setting affects the problem since our customer probably has temporary files enabled and they experience these problems always.

The problems don't seem to depend on the browser. I've tried IE9, Firefox, Chrome and Opera.

Observations

When running the applet in Windows I attached the Eclipse debugger to the applet. I suspended the execution of the applet when it was frozen and checked what it was doing. Here's what the stack looked like. "OurOwnClass.doStillMoreSomething" is the last line in our own code that was executing, and it creating a new instance of a class.

Thread [AWT-EventQueue-2] (Suspended)
    Inet6AddressImpl.lookupAllHostAddr(String) line: not available [native method]
    InetAddress$1.lookupAllHostAddr(String) line: not available
    InetAddress.getAddressFromNameService(String, InetAddress) line: not available
    InetAddress.getAllByName0(String, InetAddress, boolean) line: not available
    InetAddress.getAllByName(String, InetAddress) line: not available
    InetAddress.getAllByName(String) line: not available
    InetAddress.getByName(String) line: not available
    Handler(URLStreamHandler).getHostAddress(URL) line: not available
    Handler(URLStreamHandler).hostsEqual(URL, URL) line: not available
    Handler(URLStreamHandler).sameFile(URL, URL) line: not available
    Handler(URLStreamHandler).equals(URL, URL) line: not available
    URL.equals(Object) line: not available
    JarVerifier$VerifierCodeSource(CodeSource).equals(Object) line: not available
    JarVerifier$VerifierCodeSource.equals(Object) line: not available
    HashMap<K,V>.getEntry(Object) line: not available
    HashMap<K,V>.containsKey(Object) line: not available
    HashSet<E>.contains(Object) line: not available
    CPCallbackHandler.isTrusted(CodeSource) line: not available
    CPCallbackHandler.access$1200(CPCallbackHandler, CodeSource) line: not available
    CPCallbackHandler$ChildElement.checkResource(String) line: not available
    DeployURLClassPath$JarLoader.checkResource(String, boolean, JarEntry, JarFile, DeployURLClassPath$PathIterator) line: not available
    DeployURLClassPath$JarLoader.getResource(String, boolean, DeployURLClassPath$PathIterator) line: not available
    DeployURLClassPath.getResource(String, boolean) line: not available
    Plugin2ClassLoader$2.run() line: not available
    AccessController.doPrivileged(PrivilegedExceptionAction<T>, AccessControlContext) line: not available [native method]
    Applet2ClassLoader(Plugin2ClassLoader).findClassHelper(String) line: not available
    Applet2ClassLoader.findClass(String, boolean) line: not available
    Applet2ClassLoader(Plugin2ClassLoader).loadClass0(String, boolean, boolean) line: not available
    Applet2ClassLoader(Plugin2ClassLoader).loadClass(String, boolean, boolean) line: not available
    Applet2ClassLoader(Plugin2ClassLoader).loadClass(String, boolean) line: not available
    Applet2ClassLoader(ClassLoader).loadClass(String) line: not available
    OurOwnClass.doStillMoreSomething(String) line: 701
    OurOwnClass.doMoreSomething() line: 671
    OurOwnClass.doSometihng() line: 655
    OurOwnClass.handleTimer() line: 510
    OurOwnClass.actionPerformed(ActionEvent) line: 391
    Timer.fireActionPerformed(ActionEvent) line: not available
    Timer$DoPostEvent.run() line: not available
    InvocationEvent.dispatch() line: not available
    EventQueue.dispatchEventImpl(AWTEvent, Object) line: not available
    EventQueue.access$400(EventQueue, AWTEvent, Object) line: not available
    EventQueue$2.run() line: not available
    EventQueue$2.run() line: not available
    AccessController.doPrivileged(PrivilegedAction<T>, AccessControlContext) line: not available [native method]
    AccessControlContext$1.doIntersectionPrivilege(PrivilegedAction<T>, AccessControlContext, AccessControlContext) line: not available
    EventQueue.dispatchEvent(AWTEvent) line: not available
    EventDispatchThread.pumpOneEventForFilters(int) line: not available
    EventDispatchThread.pumpEventsForFilter(int, Conditional, EventFilter) line: not available
    EventDispatchThread.pumpEventsForHierarchy(int, Conditional, Component) line: not available
    EventDispatchThread.pumpEvents(int, Conditional) line: not available
    EventDispatchThread.pumpEvents(Conditional) line: not available
    EventDispatchThread.run() line: not available

There were several lines of code where the applet would freeze. The stack would look the same up to the first line that was in our own code (above: OurOwnClass.doStillMoreSomething), so it looked like all the freezes have the same cause. And all our lines causing this problem were creating new objects. I believe in all these cases it was also the first time that a particular class was being loaded, though I didn't check this. One of the classes was our own public class but most of them were anonymous classes with names like OurOwnClass$4 and code like:

worker = new SwingWorker<Void,Void>() {
    @Override
    public Void doInBackground() {
        // Do stuff.
        return null;
    }
};

The Java method at the top of the stack was Inet6AddressImpl.lookupAllHostAddr. The debugger indicated that this method was trying to resolve the hostname for the host that serves the applet. I think I saw the full URL of our applet JAR file somewhere, so I got the impression that the JVM was trying to access our applet JAR file again even though it should already have had it – what else was it executing?

Analysis

My best guess is that the classloader needs to do some kind of permission checking for the class about to be loaded. Perhaps the JVM is trying to check if the JAR file has been updated since it was initially loaded but the JVM hangs while it's trying to use native methods for resolving the hostname of our server. When the address resolution fails the execution of the classloader returns. All this is a bit odd since the applet can communicate with our server just fine (it gets the proxy information from the browser), but I get the feeling that the JVM doesn't use the proxy for whatever it's doing here. Does this explanation sound plausible?

Then again, I don't know how to explain the crashes and the problems getting the applet to start up in the first place. I haven't spent time trying to debug those problems so I don't know what's happening. Maybe I should try to look for hs_err_*.log files if I still go to our customer's office in the future.

It would help debugging if I had a similar setup like the customer has. I may try to set up an environment where I can only access the Internet using a HTTP proxy but I don't know if it has to be a Blue Coat proxy. I'm also unsure if I can make the proxy require authentication using the NTLM protocol and not some other method (basic, digest).

In any case, I'd appreciate help in getting rid of the freezes (and if possible, the crashes and startup problems). Could we change the code or the JAR package somehow that we could avoid these problems? Are there some JVM startup parameters that could help? Is there still something I should do to investigate the problem?

Edit

I changed the network settings for my Windows virtual machine so that it can only communciate with the host Linux, not the whole Internet. Then I set up Squid HTTP proxy on the host Linux and I told Windows to use that as the proxy. Now I get similar problems like the ones that happened at our customer's office, though not as reliably. It seems that I don't need to make the proxy require authentication.

Edit 2

I used the simulated environment and fired up Wireshark to see what the Windows guest is trying to do. Apparently it's playing a sort of ping-pong with the host: it asks the ip for our server and gets back a response "Destination unreachable (Port unreachable)" (type 3, code 3). This happens five times, and the intervals between them are 1, 1, 2 and 4 secs. Then after 4 seconds the applet goes on happily doing whatever it was doing. The intervals sum up to 12 seconds which happens to be the most common delay length. I don't know if the network at our customer's office responds that the host is unreachable or anything. Ping would timeout both over there and in this simulated setup of mine.

I'm wondering if this is a bug or a design error in the JVM. I wouldn't normally expect that instantiating a class freezes the thread for a while. I would also expect the classloader/JVM to use the proxy. Normally I would think that it's meaningful to try the hostname resolution a few times, but I'm not sure if it's the right thing to do in this context.

Edit 3

A commenter is right: it's the OS playing the ping-pong. I tried this by capturing network traffic generated by ping www.google.com and got the same behavior.

回答1:

Looking at the problematic JRE code in URLStreamHandler:

protected synchronized InetAddress getHostAddress(URL u) {
    if (u.hostAddress != null)
        return u.hostAddress;

    String host = u.getHost(); 
    if (host == null || host.equals("")) {
        return null;
    } else {
        try {
            u.hostAddress = InetAddress.getByName(host); // Hanging here
        } catch (UnknownHostException ex) {
            return null;
        } catch (SecurityException se) {
            return null;
        }
    }
    return u.hostAddress;
}

If this code fails and returns null, the parent hostsEqual method will fall back on String comparison, which is fine. So we want either UnknownHostException or SecurityException.

Looks like the firewall is dropping the DNS requests, resulting in a timeout, instead of returning NXDOMAIN (or better still, ICMP administratively prohibited).

Can you persuade the client to configure the firewall correctly? (You can test this by blocking outgoing DNS, except to the proxy, on your ubuntu box)

Another alternative would be to force the SecurityException to be thrown by InetAddress.getByName, which can happen:

if a security manager exists and its checkConnect method doesn't allow the operation

I don't know how to do this or if it's possible - can you change the permissions of the applet?

回答2:

we had the same freeze problem. where our applet application used to freeze intermittently. The root cause was "spurious re transmission", some packet seq number missing at TCP level. It was related to layer 4.

This was resolved by overriding below apache config entry with the one given next. Original value was :

SetEnvIf User-Agent ".MSIE." \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0

we overridden this values by writing this in apache_config in openshift config parameter.

SetEnvIf User-Agent ".MSIE." \ !nokeepalive !ssl-unclean-shutdown \ !downgrade-1.0 !force-response-1.0

HTTP 1.1 uses a specific header for signalling to a client when it is done transmitting data, which was not present in HTTP 1.0

来源：https://stackoverflow.com/questions/13025957/java-applet-behind-proxy-experiences-temporary-freezes-when-instantiating-new-cl

标签

java

proxy

applet

classloader