unknown error: failed to wait for extension background page to load: chrome-extension error loading an extension to Chrome Headless using Selenium

问题

I try to run chromedriver via selenium in headless mode.

IMPORTANT The code runs perfectly fine if I eliminate the following code lines (but is not headless):

chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')

This is the error I get when I try to implement the headless argument:

Traceback (most recent call last):
  File "camel_amazon_scraper.py", line 83, in <module>
    executable_path=executable_path)
  File "/home/daniel/.local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    desired_capabilities=desired_capabilities)
  File "/home/daniel/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/daniel/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/daniel/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/daniel/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: failed to wait for extension background page to load: chrome-extension://jkompbllimaoekaogchhkmkdogpkhojg/_generated_background_page.html
from unknown error: page could not be found: chrome-extension://jkompbllimaoekaogchhkmkdogpkhojg/_generated_background_page.html

This are the lines 81, 82 and 83

chrome_options.add_extension(extension_path)
driver = webdriver.Chrome(options=chrome_options,
                          executable_path=executable_path)

This is the code (the crhomedriver execution parts):

from selenium import webdriver
from selenium.webdriver import Chrome
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.expected_conditions import presence_of_element_located

import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))

chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
log_path = os.path.join(BASE_DIR, 'cronJobChromeDriver.log')

executable_path = os.path.join(BASE_DIR, 'chromedriver_linux64/chromedriver')
extension_path = os.path.join(
    BASE_DIR, 'chromedriver_linux64/extension_2_8_9_0.crx')
print('executable_path', executable_path)

回答1:

Chrome does not support headless, but apparently Firefox does. Some relevant discussions:

https://sqa.stackexchange.com/questions/32611/selenium-chromedriver-headless-chrome-failed-to-wait-for-extension-backgro
Is it possible to run Google Chrome in headless mode with extensions?

回答2:

The bottom line is, No, google-chrome-headless doesn't supports extensions.

In the one of his comment, alexclarke@chromium.org mentioned:

I realize a lot of folks would like to use extensions with headless but unfortunately that's a large project which we have /no plans to do/. The problem is Headless Chromium is a content embedder which means it doesn't have access to anything from other content embedders such as chrome and unfortunately extensions are a chrome feature.

In another comment he further added, if you're using Selenium through DevTools you can build a proxy. Next you can filter URLs and modify headers via Network.setRequestInterception and Network.continueInterceptedRequest.

Reference

You can find a relevant detailed discussion in:

Is it possible to run Google Chrome in headless mode with extensions?

来源：https://stackoverflow.com/questions/62344970/unknown-error-failed-to-wait-for-extension-background-page-to-load-chrome-exte

标签

python

selenium

google-chrome-extension

selenium-chromedriver

google-chrome-headless