Selenium webdriver.Remote driver does not work with tor proxy(webdriver.Chrome does)

Question

I\'m trying to use socks5 proxy on my remote driver which is hosted as a docker container on port 4444.

here is code-sample:

from selenium import we         


        

Answer 1

I suggest you to add the configuration of your proxy in docker config.json file:

{
 "proxies":
 {
   "default":
   {
     "httpProxy": "socks5://127.0.0.1:9050",
     "httpsProxy": "socks5://127.0.0.1:9050"
   }
 }
}

and remove it from your python script

Answer 2

For macOS and Windows you can use host.docker.internal to access local host from container:

from selenium import webdriver

opts = webdriver.ChromeOptions()
opts.add_argument("--no-sandbox")
opts.add_argument("--disable-dev-shm-usage")
opts.add_argument("--proxy-server=socks5://host.docker.internal:9050")
driver = webdriver.Remote(command_executor="http://localhost:4444/wd/hub", desired_capabilities=opts.to_capabilities())

driver.get("http://jsonip.com/")
print(driver.find_element_by_css_selector("html").text)
driver.quit()

Here is how selenium hub works with tor proxy. You can create network in docker, attach containers to it and then use container name to proxy host:

docker network create mynetwork
docker run -it -p 8118:8118 -p 9050:9050 --name tor-proxy -d dperson/torproxy
docker run -d -p 4444:4444 --name selenium-hub -v /dev/shm:/dev/shm selenium/standalone-chrome:3.141.59-yttrium
docker network connect mynetwork hub
docker network connect mynetwork tor-proxy

Here same example using docker compose:

version: '3.5'

services:

  tor-proxy:
    image: dperson/torproxy
    container_name: tor-proxy
    ports:
      - "8118:8118"
      - "9050:9050"
    networks:
      - mynetwork

  selenium-hub:
    image: selenium/standalone-chrome:3.141.59-yttrium
    container_name: selenium-hub
    ports:
      - "4444:4444"
    networks:
      - mynetwork

networks:
  mynetwork:
    name: mynetwork
    driver: bridge

Python Code:

from selenium import webdriver

opts = webdriver.ChromeOptions()
opts.add_argument("--no-sandbox")
opts.add_argument("--disable-dev-shm-usage")
opts.add_argument("--proxy-server=socks5://tor-proxy:9050")
driver = webdriver.Remote(command_executor="http://localhost:4444/wd/hub", desired_capabilities=opts.to_capabilities())

driver.get("http://jsonip.com/")
print(driver.find_element_by_css_selector("html").text)
driver.quit()

Result:

{"ip":"18.27.197.252","about":"https://jsonip.com/about","Pro!":"http://getjsonip.com","Get Notifications": "https://jsonip.com/notify"}

Process finished with exit code 0

Run same code again:

{"ip":"178.165.72.177","about":"https://jsonip.com/about","Pro!":"http://getjsonip.com","Get Notifications": "https://jsonip.com/notify"}

Process finished with exit code 0