Selenium with chromedriver gives different results based on “headless” argument

Question

I have been playing around with Selenium + Chromedriver and I noticed I get different results based on if headless is enabled or disabled. After some investigation I found out that "headless" does not include the Accept-Language header.

Is there anyway to manually add this in the headers?

Answer 1

That's what Google chrome does. One way around it would be to use a proxy and modify the headers for you. Or you can use the Firefox driver as that driver does not send different headers when using the headless option.

Answer 2

Ideally, using and not using the --headless option shouldn't have any major effect on the elements within the DOM Tree getting rendered but may have a significant difference as far as the Viewport is concerned.

As an example, when ChromeDriver/Chrome is initialized along with the --headless option the default Viewport is

width = 800px, height = 600px 

Where as when ChromeDriver/Chrome is initialized without the --headless option the default Viewport is:

width = 1050px, height = 708px

• Example Code (Python based):

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  options = webdriver.ChromeOptions()
  options.add_argument("--headless")
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
  driver.get("https://www.google.com/")
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.NAME, "q")))
  print ("Headless Chrome Initialized")
  size = driver.get_window_size()
  print("Window size: width = {}px, height = {}px".format(size["width"], size["height"]))
  driver.quit()
  options = webdriver.ChromeOptions()
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
  driver.get("https://www.google.com/")
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.NAME, "q")))
  print ("Chrome Initialized")
  size = driver.get_window_size()
  print("Window size: width = {}px, height = {}px".format(size["width"], size["height"]))
  

• Console Output:

  Headless Chrome Initialized
  Window size: width = 800px, height = 600px
  Chrome Initialized
  Window size: width = 1050px, height = 708px
  

So it can be concluded that with --headless option ChromeDriver/Chrome opens the session with reduced Viewport and hence the number of elements identified are less.

---

Solution

While using ChromeDriver/Chrome to initiate a Browsing Instance always open in maximized mode:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument("--headless")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.google.com/")