how to resume an interrupted download - part 2

Question

This is a continuation of my previous question which I posted when I wasn\'t a registered user. As a refresher, I\'m trying to resume the downloading of a large file from m

Answer 1

To resume a download, you need to send not only the Range request header, but also the If-Range request header which should contain either the unique file identifier or the file modification timestamp.

If the server returns an ETag response header on the initial download, then you should use it in the If-Range header of the subsequent resume requests. Or if it returns a Last-Modified response header, then you should use it in the If-Range request header instead.

Looking at your logs, the server has sent a Last-Modified response header. So you should send it back along in an If-Range header of the resume request.

// Initial download.
String lastModified = connection.getHeaderField("Last-Modified");

// ...

// Resume download.
connection.setRequestProperty("If-Range", lastModified); 

The server will use this information to verify if you're requesting exactly the same file.

Answer 2

I managed to implement http resume download on my client Java application, trick was to use "If-Range:original_ETag" request header as mentioned in an accepted answer. Here is a full header debug information it may help you understand how resume works.

Normal reply from server, file is read from the start to the end. ETag value is W/"filesize_bytes-modified_utc" metadata from the disk file. It could be something else but thats what Tomcat uses and usually is best for regular file content.

HTTP/1.1 200 OK 
ETag: W/"19097900-1410863319978" 
Last-Modified: Tue, 16 Sep 2014 10:28:39 GMT 
Content-Length: 19097900 
Content-Type: application/octet-stream 
Accept-Ranges: bytes 
Server: Apache-Coyote/1.1 
Date: Wed, 17 Sep 2014 10:39:52 GMT 

Client reads a given chunk from the file by adding Range and If-Range headers. Byte range can be open so server should read starting from given index to the end. Note Content-Length is not the total length of file but this chunk of bytes.

Range: bytes=10000000-
If-Range: W/"19097900-1410863319978"

HTTP/1.1 206 Partial Content 
ETag: W/"19097900-1410863319978" 
Last-Modified: Tue, 16 Sep 2014 10:28:39 GMT 
Content-Length: 9097900 
Content-Type: application/octet-stream 
Accept-Ranges: bytes 
Content-Range: bytes 10000000-19097899/19097900 
Server: Apache-Coyote/1.1 
Date: Wed, 17 Sep 2014 18:12:36 GMT 

If remote file was changed then server should not return 206-PartialContent but regular 200-OK reply. Server should use If-Range value to check for consistency. Some clients like VLCPlayer puts Range field without If-Range value.

When client is making an initial request without Range+If-Range headers it may add "If-Modified-Since: modified_http_date" header. Server may return 304-NotModified status and client is done. If client gives byteStartIdx>=filesize then server returns 416-RequestedRangeNotSatisfiable.

Additional debug dumps to clarify how http-resume works.

Range=bytes=0-0
If-Range=W/"19097900-1410863319978"

HTTP/1.1 206 Partial Content 
ETag=W/"19097900-1410863319978" 
Date=Fri, 19 Sep 2014 12:53:36 GMT 
Content-Length=1 
Last-Modified=Tue, 16 Sep 2014 10:28:39 GMT 
Content-Type=application/octet-stream 
Accept-Ranges=bytes 
Server=Apache-Coyote/1.1 
Content-Range=bytes 0-0/19097900 
- - - - -

Range=bytes=19097800-29097800
If-Range=W/"19097900-1410863319978"

HTTP/1.1 206 Partial Content 
ETag=W/"19097900-1410863319978" 
Date=Fri, 19 Sep 2014 12:59:24 GMT 
Content-Length=100 
Last-Modified=Tue, 16 Sep 2014 10:28:39 GMT 
Content-Type=application/octet-stream 
Accept-Ranges=bytes 
Server=Apache-Coyote/1.1 
Content-Range=bytes 19097800-19097899/19097900 
- - - - - - - - 

Range=bytes=19097899-19097899
If-Range=W/"19097900-1410863319978"

HTTP/1.1 206 Partial Content 
ETag=W/"19097900-1410863319978" 
Date=Fri, 19 Sep 2014 13:01:47 GMT 
Content-Length=1 
Last-Modified=Tue, 16 Sep 2014 10:28:39 GMT 
Content-Type=application/octet-stream 
Accept-Ranges=bytes 
Server=Apache-Coyote/1.1 
Content-Range=bytes 19097899-19097899/19097900 

Answer 3

Seems that problem is calling

input = new BufferedInputStream(url.openStream(), 8192);

instead of

input = new BufferedInputStream(connection.getInputStream(), 8192);

url.openStream() makes another call to openConnection() WITHOUT the range property.