Why wget ignores query string in the url?

Question

I want to use wget to download the following 18 html files:

http://www.ted.com/talks/quick-list?sort=date&order=desc&page=18  
http://www.ted.com/talks/q         


        

Answer 1

1. Store your list of URLs in a file (each URL in a separate line!!):

   echo "http://www.ted.com/talks/quick-list?sort=date&order=desc&page=18 http://www.ted.com/talks/quick-list?sort=date&order=desc&page=17 ... " > wget_filelist.txt

2. Call wget to retrieve the stuff:

   wget -i wget_filelist.txt

Answer 2

& is a special character in most shell environments, you can use double quotes to quote the URL to pass the whole thing in as the parameter to wget:

wget "http://www.ted.com/talks/quick-list?sort=date&order=desc&page=18"

Answer 3

Special case: There is still a problem with wget "URL" format, even though it solved the problem of & it can't pass ! symbol.

Solution: Single quote instead of double quote for the URL will fix this, example:

wget 'https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-G.798-201712-I!!PDF-E&type=items'

As shown in above example, it works for me which contains both & and ! symbols in it. I am not sure if it's an excepted solution for all platform (ie., official POSIX shell).

Bonus: Further we can use wget -c 'URL', so that, in case there is a failure in one go and we don't need to start from beginning.