HttpURLConnection does not read the whole response

Question

I use HttpURLConnection to do HTTP POST but I dont always get back the full response. I wanted to debug the problem, but when I step through each line it worked. I thought it mu

Answer 1

If you are reading the whole Message at once you can compare the isr.available() to the expected content lenght. This is how I did it:

public byte[] readData(HttpURLConnection conn)
        throws IOException, InterruptedException {
    String _connlen = conn.getHeaderField("Content-Length");
    int connlen = Integer.parseInt(_connlen);
    InputStream isr = null;
    byte[] bytes = new byte[connlen];

    try {
        isr = conn.getInputStream();

        //security count that it doesn't begin to hang
        int maxcounter = 0;
        //wait till all data is avalibal, max 5sec
        while((isr.available() != connlen) && (maxcounter  < 5000)){
            Thread.sleep(1);
            maxcounter++;
        }
        //Throw if not all data could be read
        if(maxcounter >= 5000)
            throw new IllegalAccessError(); 

        //read the data         
        if(isr.read(bytes, 0, connlen) < 0)
            throw new IllegalAccessError();     


    } finally {
        if (isr != null)
            isr.close();
    }

    return bytes;
}

Answer 2

Maybe I missed it, but what's the datatype of "input" in your code? Something that's strange about InputStreams in general is that the read( ... ) methods tend to block until data is available, then return only that data. You'll actually need to keep reading from your InputStream and appending to a ByteArrayInputStream or some other structure until you explicitly force a EOFException.

Answer 3

I think your problem is in this line:

while (is.available() > 0) {

According to the javadoc, available does not block and wait until all data is available, so you might get the first packet and then it will return false. The proper way to read from an InputStream is like this:

int len;
byte[] buffer = new byte[4096];
while (-1 != (len = in.read(buffer))) {
  bos.write(buffer, 0, len);
}

Read will return -1 when there nothing left in the inputstream or the connection is closed, and it will block and wait for the network while doing so. Reading arrays is also much more performant than using single bytes.