Pause and resume downloading with pycurl

问题

I'm writing a code to download files using pycurl. So I want to know that is there a possibility to pause my download and then resume it from wherever it's been paused? Does pycurl support these features or is there any other library which support pause and resume?

回答1:

import os
import pycurl
import sys


def progress(total, existing, upload_t, upload_d):
    existing = existing + os.path.getsize(filename)
    try:
        frac = float(existing)/float(total)
    except:
        frac = 0
    sys.stdout.write("\r%s %3i%%" % ("File downloaded - ", frac*100))

url = raw_input('Enter URL to download folder/file: ')
filename = url.split("/")[-1].strip()


def test(debug_type, debug_msg):
    print "debug(%d): %s" % (debug_type, debug_msg)

c = pycurl.Curl()
c.setopt(pycurl.URL, url)
c.setopt(pycurl.FOLLOWLOCATION, 1)
c.setopt(pycurl.MAXREDIRS, 5)

# Setup writing
if os.path.exists(filename):
    f = open(filename, "ab")
    c.setopt(pycurl.RESUME_FROM, os.path.getsize(filename))
else:
    f = open(filename, "wb")

c.setopt(pycurl.WRITEDATA, f)

#c.setopt(pycurl.VERBOSE, 1) 
c.setopt(pycurl.DEBUGFUNCTION, test)
c.setopt(pycurl.NOPROGRESS, 0)
c.setopt(pycurl.PROGRESSFUNCTION, progress)
try:
    c.perform()
except:
    pass

Possible to download file for example - *.tar, .dmg, ,exe, jpg etc .

Used pycurl for resume download.

Testing of this code as:

Run this file on terminal like:

Here is the log on terminal,

anupam@anupampc:~/Documents$ python resume_download.py 

Enter URL to download folder/file: http://nightly.openerp.com/6.0/6.0/openerp-allinone-setup-6.0-20110625-r3451.exe 

File downloaded - 199%

If you stop this download by pressing Ctrl + C and starts process to download file/folder , it start from where it was stopped.

回答2:

If you stop the thread and close the connection, you can use HTTP Content-Range to continue your request where you left off. Just find out how many bytes you have already, and use RESUME_FROM to start from there:

import pycurl

starting_point = 999 # calculate this

url="http://test-url"
curl = pycurl.Curl()
curl.setopt(curl.URL, url)
curl.setopt(curl.RESUME_FROM, starting_point)
curl.perform()
curl.close()

来源：https://stackoverflow.com/questions/13775892/pause-and-resume-downloading-with-pycurl

标签

python

download

pycurl