using proxy with scrapy-splash

前端 未结 1 864
北海茫月
北海茫月 2021-01-01 04:10

I\'m trying to use proxy (proxymesh) alongside scrapy-splash. I have following (relevant) code

PROXY = \"\"\"splash:on_request(function(request)
    request:         


        
相关标签:
1条回答
  • 2021-01-01 04:28

    You should add 'proxy' argument to SplashRequest object.

    def start_requests(self):
        for url in self.start_urls:
            print url
            yield SplashRequest(url, self.parse,
                endpoint='execute',
                args={'wait': 5,
                      'lua_source': PROXY,
                      'js_source': 'document.body',
                      'proxy': 'http://proxy_ip:proxy_port'}
    
    0 讨论(0)
提交回复
热议问题