Get size of a file before downloading in Python

Question

I\'m downloading an entire directory from a web server. It works OK, but I can\'t figure how to get the file size before download to compare if it was updated on the server

Answer 1

Also if the server you are connecting to supports it, look at Etags and the If-Modified-Since and If-None-Match headers.

Using these will take advantage of the webserver's caching rules and will return a 304 Not Modified status code if the content hasn't changed.

Answer 2

A requests-based solution using HEAD instead of GET (also prints HTTP headers):

#!/usr/bin/python
# display size of a remote file without downloading

from __future__ import print_function
import sys
import requests

# number of bytes in a megabyte
MBFACTOR = float(1 << 20)

response = requests.head(sys.argv[1], allow_redirects=True)

print("\n".join([('{:<40}: {}'.format(k, v)) for k, v in response.headers.items()]))
size = response.headers.get('content-length', 0)
print('{:<40}: {:.2f} MB'.format('FILE SIZE', int(size) / MBFACTOR))

Usage

$ python filesize-remote-url.py https://httpbin.org/image/jpeg
...
Content-Length                          : 35588
FILE SIZE (MB)                          : 0.03 MB