Python progress bar and downloads

Question

I have a python script that launches a URL that is a downloadable file. Is there some way to have python use commandline to display the download progress as oppose to launch

Answer 1

Updated for your sample url:

I've just written a super simple (slightly hacky) approach to this for scraping pdfs off a certain site. Note, it only works correctly on unix based systems (linux, mac os) as powershell does not handle "\r"

import requests

link = "http://indy/abcde1245"
file_name = "download.data"
with open(file_name, "wb") as f:
    print "Downloading %s" % file_name
    response = requests.get(link, stream=True)
    total_length = response.headers.get('content-length')

    if total_length is None: # no content length header
        f.write(response.content)
    else:
        dl = 0
        total_length = int(total_length)
        for data in response.iter_content(chunk_size=4096):
            dl += len(data)
            f.write(data)
            done = int(50 * dl / total_length)
            sys.stdout.write("\r[%s%s]" % ('=' * done, ' ' * (50-done)) )    
            sys.stdout.flush()

It uses the requests library so you'll need to install that. This outputs something like the following into your console:

>Downloading download.data

>[=============                            ]

The progress bar is 52 characters wide in the script (2 characters are simply the [] so 50 characters of progress). Each = represents 2% of the download.

Answer 2

I'm surprised that tqdm has not been suggested!

Answer 3

#ToBeOptimized - Baseline If you would like to puzzle your brain and hand craft the logic

# Define Progress Bar function

def print_progressbar(total,current,barsize=60):
    progress=int(current*barsize/total)
    completed= str(int(current*100/total)) + '%'
    print('[' , chr(9608)*progress,' ',completed,'.'*(barsize-progress),'] ',str(i)+'/'+str(total), sep='', end='\r',flush=True)

# Sample Code

total= 6000
barsize=60
print_frequency=max(min(total//barsize,100),1)
print("Start Task..",flush=True)
for i in range(1,total+1):
  if i%print_frequency == 0 or i == 1:
    print_progressbar(total,i,barsize)
print("\nFinished",flush=True)

# Snapshot of Progress Bar :

Below lines are for illustrations only. In command prompt you will see single progress bar showing incremental progress.

[ 0%............................................................] 1/6000

[██████████ 16%..................................................] 1000/6000

[████████████████████ 33%........................................] 2000/6000

[██████████████████████████████ 50%..............................] 3000/6000

[████████████████████████████████████████ 66%....................] 4000/6000

[██████████████████████████████████████████████████ 83%..........] 5000/6000

[████████████████████████████████████████████████████████████ 100%] 6000/6000

Good Luck and Enjoy!

Answer 4

You can use the 'clint' package (written by the same author as 'requests') to add a simple progress bar to your downloads like this:

from clint.textui import progress

r = requests.get(url, stream=True)
path = '/some/path/for/file.txt'
with open(path, 'wb') as f:
    total_length = int(r.headers.get('content-length'))
    for chunk in progress.bar(r.iter_content(chunk_size=1024), expected_size=(total_length/1024) + 1): 
        if chunk:
            f.write(chunk)
            f.flush()

which will give you a dynamic output which will look like this:

[################################] 5210/5210 - 00:00:01

It should work on multiple platforms as well! You can also change the bar to dots or a spinner with .dots and .mill instead of .bar.

Enjoy!