问题
I am using BeautifulSoup and using UserAgent while making request. Amazon is blocking my calls though I did add a sleep method to avoid it. Is there anyway to deal with it? I know there's an API available but I doubt I would get what I am looking for.
What I want to get product details based on ASIN and it returns all price offers by different sellers using Amazon Prime shipping option, the example URL given here.
回答1:
Is there anyway to deal with it?
Yes... you comply with their acceptable use policy.
If it's not available from an API, you're not authorized to scrape it.
Even if you successfully scrape it, you're still not authorized to use it.
This license does not include any resale or commercial use of any Amazon Service, or its contents; any collection and use of any product listings, descriptions, or prices; any derivative use of any Amazon Service or its contents; any downloading, copying, or other use of account information for the benefit of any third party; or any use of data mining, robots, or similar data gathering and extraction tools.
https://www.amazon.com/gp/help/customer/display.html/ref=ap_frn_condition_of_use?ie=UTF8&nodeId=508088
来源：https://stackoverflow.com/questions/21449431/amazon-scraping-returns-503