问题
Context: I am using Python, selenium + Geckodriver. Firefox webbrowser automation and I have problem on a private proxy authentification. I cannot manage to configure my socks5 proxy username and password. Thus, Firefox shows up but connection doesnt work.
The simple task that I want to do is :
- I want to use selenium(automated Firefox) with a proxy (SOCKS5 proxy with password)
- Checking my IP on a website to see if the proxy is working
What I noted so far is:
- I am using Firefox V76.0.1 , Geckodriver and Python3
- The User / Password box that should normally pop up when u choose doesn't pop up, this could be a problem with regards to the Firefox version and it could be fixed by overriding some settings in : about:setings. A solution could be to make this popup work and write some User password auth - see login function below. Honestly , it is unstable I prefer to find a solution that can work without this popup.
- The only stable way to manage socks5 proxies (regardless of the popup authentification) is using an AddOn called FoxyProxy.
- I cannot change this (Foxy Proxy AddOn) and all I can change in python with regards to Firefox are the settings that you can acess in Firefox using the URL (about:config)
- Simple non password protected proxies are not a problem but SOCKS5 proxies with username and pasword are a problem
- Also: wondering if it is possible to override Foxy Proxy AddOn and add options that can be accessible in Firefox (about:config) ( Honestly I am not a Firefox plugin developper so maybe it is possible!? )
So the current code is, replace HOST/USERNAME/PORT/PASSWORD with socks5 proxy data :
from selenium import webdriver
from base64 import b64encode
from selenium import webdriver
def login(browser):
alert=browser.switch_to_alert()
alert.send_keys("username"+webdriver.common.keys.Keys.TAB+"password")
alert.accept()
proxy = {'host': HOST, 'port': PORT, 'usr': USERNAME, 'pwd': PASSWORD}
fp = webdriver.FirefoxProfile()
fp.set_preference('network.proxy.type', 1)
fp.set_preference('network.proxy.http', proxy['host'])
fp.set_preference('network.proxy.http_port', int(proxy['port']))
fp.set_preference('network.proxy.no_proxies_on', 'localhost, 127.0.0.1')
fp.set_preference("network.proxy.socks_version", 5)
credentials = '{usr}:{pwd}'.format(**proxy)
fp.set_preference("network.proxy.type", 1)
fp.set_preference("network.proxy.socks_version", 5)
fp.set_preference("network.proxy.socks", HOST)
fp.set_preference("network.proxy.socks_port", PORT)
fp.set_preference("network.http.use-cache", False)
driver = webdriver.Firefox(fp)
# normally a proxy login popup should appear and we should call the login function here!!
driver.get('https://wtfismyip.com')
Doesnt work, Any Ideas ?
回答1:
To add foxyproxy as an extension:
driver = webdriver.Firefox(executable_path="path to geckodriver", options=options)
driver.install_addon("path to .xpi file of the extension", temporary=None)
driver.switch_to.window(driver.window_handles[1])`
So it should open a webpage where you can set your proxy. Now, to set the proxy, first i define a function that lets me insert all the proxy's stuff (host, port, username, password)
def set_proxy(address, port, usernamex, passwordx):
proxy_address_xpath = '//*[@id="proxyAddress"]'
proxy_address_box = driver.find_element_by_xpath(proxy_address_xpath)
proxy_address_box.send_keys(address)
proxy_port_xpath = '//*[@id="proxyPort"]'
proxy_port_box = driver.find_element_by_xpath(proxy_port_xpath)
proxy_port_box.send_keys(port)
proxy_username_xpath = '//*[@id="proxyUsername"]'
proxy_username_box = driver.find_element_by_xpath(proxy_username_xpath)
proxy_username_box.send_keys(usernamex)
proxy_pass_xpath = '//*[@id="proxyPassword"]'
proxy_pass_box = driver.find_element_by_xpath(proxy_pass_xpath)
proxy_pass_box.send_keys(passwordx)
save_xpath = '/html/body/div[2]/div[2]/button[4]'
save_box = driver.find_element_by_xpath(save_xpath)
save_box.click()
then, with the following code, I set the proxy and select it, so that when i navigate it will use the proxy
# go to the page where you can add the proxies
xpath_back ="/html/body/div[2]/div[2]/button"
back_box = driver.find_element_by_xpath(xpath_back)
back_box.click()
# add proxy
add_xpath = "/html/body/div[4]/div/nav/a[1]"
add_box = WebDriverWait(driver, timeout=10).until(lambda x: x.find_element_by_xpath(add_xpath))
add_box.click()
# select proxy type (in my case socks5)
type_xpath = '//*[@id="proxyType"]'
type_box = driver.find_element_by_xpath(type_xpath)
type_box.click()
socks5_xpath = '/html/body/div[2]/div[1]/div[2]/div[1]/select/option[3]'
socks5_box = driver.find_element_by_xpath(socks5_xpath)
socks5_box.click()
# write the proxy stuff
set_proxy('socks5', 'host', 'port', 'pass')
# select the proxy you just saved (in this case it selects the first one)
change_xpath = '//*[@id="mode"]'
change_box = WebDriverWait(driver, timeout=10).until(lambda x: x.find_element_by_xpath(change_xpath))
change_box.click()
select_xpath = '/html/body/div[4]/div/div/div[1]/select/option[2]'
select_box = driver.find_element_by_xpath(select_xpath)
select_box.click()
then, to navigate, be sure to do that in a new windows or switch to the first one using
driver.switch_to.window(driver.window_handles[0])
so that you can keep the foxyproxy window for example to change proxy.
If you want to use other ways, check How to set proxy authentication (user & password) using Python + Selenium
来源:https://stackoverflow.com/questions/61879050/cannot-use-private-socks5-proxy-with-seleniumfirefox-python