cURL - Load a site with CloudFlare protection

Question

I have a site which I want to load using cURL. However, the site I\'m talking about is using cloudflare protection. While on cURL request => it says \'checking your browser,

Answer 1

That would be CloudFlare's I'm Under Attack Mode --> http://blog.cloudflare.com/introducing-im-under-attack-mode

If you are the owner of the site you should whitelist your own IP address. If you're a visitor to the site either

1.) The website owner has the feature enabled on their own website in which case it's intentional and there is no way around that as they are protecting their website or

2.) You are loading the site too often -- and you're triggering CloudFlare's DDOS protection.

p.s. Disclaimer: I work for CloudFlare.

Answer 2

You can get around the in-under-attack-mode by selecting 'Copy as cUrl' in the network section under developer tools of any major browser.

This copies all the required cookies so your curl can be 'authenticated'. How long these cookies last i would assume would be up to cloudflare.

SideNote: The curl must be run on the same IP as you were loading the site with. Also for any real test/developing environment I would suggest selenium and/or phantomjs which are alot better and easily scriptable languages for website testing.

curl 'https://ilikeurls.net/ourpage.php?do=command' \
    -H 'Host: ilikeurls.net' \
    -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0' \
    -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
    -H 'Accept-Language: en-US,en;q=0.5' \
    -H 'Referer: https://ilikeurls.net/outpage.php' \                                                    
    -H 'Cookie: all required cookies will appear here' \
    -H 'Connection: keep-alive'\
    --compressed

Though you might be better off using selenium or phantomjs to script out these tests.