How to bulk download files from the internet archive

一曲冷凌霜 提交于 2020-04-18 00:43:59

问题


I checked the original site of the internet archive and they mentioned there a couple of steps to follow, which included the use of the wget utility using Cygwin over windows, I followed the steps above, I made an advanced search and extracted the CSV file, converted it to .txt and then tried to run the following commands

wget -r -H -nc -np -nH --cut-dirs=1 -A .pdf,.epub -e robots=off -l1 -i ./itemlist.txt -B 'http://archive.org/download/

The emulator gets stuck afterwards and no log message or even an error message appears indicating any practical progress, I want to know what wrong have I done so far.


回答1:


After Some time I figured out how to resolve this matter, the commands posted in the internet archive help blog are general commands posted to help use the wget utility , the commands we will need right here are simply those which follow

--cutdirs=1
-A .pdf,.epub
-e robots=off
-i ./itemlist.txt

and of course the url source:

B- 'archive.org/download/'


来源:https://stackoverflow.com/questions/60881429/how-to-bulk-download-files-from-the-internet-archive

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!