downloading with curl using multiple connections

感情迁移 提交于 2020-01-13 08:34:32

问题


I'm trying to tranfer a multi-gb file from one server to another; problem is RTT is 150ms+. Ive already tried using aria2 but its limited to 16 connections, lftp doesn't have any protection against stalled transfers.

I'm wondering if its possible to download one file with multiple connections using curl cli.


回答1:


No, the curl tool has no such ability.

(oh and btw, a large RTT is very rarely the explanation to why a plain TCP transfer is slow)




回答2:


It's possible. Fetch the total file size with -I option in curl.

Then you can fork many process in a shell, every curl connection with a different Content-Length header to download different part of the file.

After all the tasks finish, then merge all the download slices to a big file.

I have written a simple script and it's available here mcurl.sh, with -s option you can specify how many tasks you create to download the big file.



来源:https://stackoverflow.com/questions/8378899/downloading-with-curl-using-multiple-connections

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!