Java URLConnection : how can I find out the size of a web file?

Question

I\'m working on a project for school, and I\'m implementing a tool which can be used to download files from the web ( with a throttling option ). The thing is, I\'m gonna ha

Answer 1

Any HTTP response is supposed to contain a Content-Length header, so you could query the URLConnection object for this value.

//once the connection has been opened
List values = urlConnection.getHeaderFields().get("content-Length")
if (values != null && !values.isEmpty()) {

    // getHeaderFields() returns a Map with key=(String) header 
    // name, value = List of String values for that header field. 
    // just use the first value here.
    String sLength = (String) values.get(0);

    if (sLength != null) {
       //parse the length into an integer...
       ...
    }

It might not always be possible for a server to return an accurate Content-Length, so the value could be inaccurate, but at least you would get some usable value most of the time.

update: Or, now that I look at the URLConnection javadoc more completely, you could just use the getContentLength() method.

Answer 2

As mentioned, URLConnection's getContentLengthLong() is your best bet, but it won't always give a definite length. That's because the HTTP protocol (and others that could be represented by a URLConnection) doesn't always convey the length.

In the case of HTTP, the length of dynamic content typically isn't known in advance—when the content-length header would normally be sent. Instead, another header, transfer-encoding, specifies that a "chunked" encoding is used. With chunked encoding, the length of the entire response is unspecified, and the response is sent back in pieces, where the size of each piece is specified. In practice, the server buffers output from the servlet. Whenever the buffer fills up, another chunk is sent. Using this mechanism, HTTP could actually start streaming a response of infinite length.

If a file is larger than 2 Gb, its size can't be represented as an int, so the older method, getContentLength() will return -1 in that case.

Answer 3

As @erickson said, sometimes there is header "Transfer-Encoding: chunked", instead of "Content-Length: " and of course you have null value for length.

About the available() method - nobody can guarantee to you that it will return proper value, so I recommend you to not use it.

Answer 4

You'll want to use the content length (URLConnection.getContentLength()). Unfortunately, this won't always be accurate, or may not always be provided, so it's not always safe to rely on it.

Answer 5

    //URLConnection connection

private int FileSize(String url) {

 // this is the method and it get the url as a parameter.

       // this java class will allow us to get the size of the file.

        URLConnection con; 

         // its in a try and catch incase the url given is wrong or invalid

        try{ 

            // we open the stream

            con = new URL(url).openConnection()

            return con.getContentLength(); 
        }catch (Exception e){

            e.printStackTrace();

            // this is returned if the connection went invalid or failed.

            return 0; 
        }
    }

Answer 6

Using a HEAD request, i got my webserver to reply with the correct content-length field which otherwise was empty. I don't know if this works in general but in my case it does:

    private int tryGetFileSize(URL url) {
        HttpURLConnection conn = null;
        try {
            conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("HEAD");
            conn.getInputStream();
            return conn.getContentLength();
        } catch (IOException e) {
            return -1;
        } finally {
            conn.disconnect();
        }
    }