Selenium WebDriver not able to find elements which are not present in the Page Source but present in HTML when seen through Developer Tools

问题

I am using Selenium Web Driver HtmlUnitDriver for searching elements on a web page. I am able to search only those elements that are visible in the Page Source. However I can see these elements details using the Internet Explorer Developer Tools (F12). When I use these details (id/name/XPath) using Selenium, it throws a org.openqa.selenium.NoSuchElementException. The code I have written is :

public static void main(String[] args) {

        WebDriver driver = new HtmlUnitDriver(){
            @Override
            protected WebClient modifyWebClient(WebClient client) {

                DefaultCredentialsProvider credentialsProvider = new DefaultCredentialsProvider();
                credentialsProvider.addNTLMCredentials("username", "password", null, -1, "localhost", "domain");
                client.setCredentialsProvider(credentialsProvider);
                return client;
            }
        };

        driver.get("URL");

        System.out.println(driver.getPageSource());
        WebElement myDynamicElement = (new WebDriverWait(driver, 20)).until(ExpectedConditions.presenceOfElementLocated(By.id("ppm_timesheet")));

    }

PAGE SOURCE

<?xml version="1.0" encoding="UTF-8"?>
    <html>
    <head>
    <meta name="application-name" content="CONTENT"/>
    <meta name="upk-namespace" content="en"/>
    <meta http-equiv="X-UA-Compatible" content="IE=8"/>
    <link rel="SHORTCUT ICON" href="ui/uitk/images/shortcut.ico"/>
    <title>
      Title
    </title>
    <link rel="stylesheet" href="ui/uitk/css/min.css"/>
    <script type="text/javascript" src="ui/ext/ext.min.js">
    </script>
    <script type="text/javascript">
    //<![CDATA[
    require( {baseUrl: "ui"},[ "uif/js/main.min" ] );
    //]]>
    </script>
  </head>
  <body>
    <div id="ppm">
    </div>
  </body>
</html>

If I search the element "ppm", it is successful which is present in page source. When I search the element "ppm_timesheet" I am getting following exception maybe because this element is not present in the page source. But when I press F12 in Internet Explorer and select this element in developer tools, this element is present there in HTML:

<button title="Time" class="class" id="ppm_timesheet" type="submit" jQuery171013988872804261454="13" alt="Time">

The full exception is:

Exception in thread "main" org.openqa.selenium.TimeoutException: Timed out after 20 seconds
Build info: version: '2.8.0', revision: '14056', time: '2011-10-06 12:41:26'
System info: os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1 build 7601 Service Pack 1', java.version: '1.6.0'
Driver info: driver.version: unknown
    at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:220)
    at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:188)
    at com.auto.Automation.main(Automation.java:32)
Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate element with ID: ppm_timesheet
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.8.0', revision: '14056', time: '2011-10-06 12:41:26'
System info: os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1 build 7601 Service Pack 1', java.version: '1.6.0'
Driver info: driver.version: HtmlUnitDriver
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElementById(HtmlUnitDriver.java:685)
    at org.openqa.selenium.By$ById.findElement(By.java:210)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver$5.call(HtmlUnitDriver.java:1222)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver$5.call(HtmlUnitDriver.java:1)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.implicitlyWaitFor(HtmlUnitDriver.java:969)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElement(HtmlUnitDriver.java:1219)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElement(HtmlUnitDriver.java:396)
    at org.openqa.selenium.support.ui.ExpectedConditions.findElement(ExpectedConditions.java:289)
    at org.openqa.selenium.support.ui.ExpectedConditions.access$0(ExpectedConditions.java:287)
    at org.openqa.selenium.support.ui.ExpectedConditions$3.apply(ExpectedConditions.java:86)
    at org.openqa.selenium.support.ui.ExpectedConditions$3.apply(ExpectedConditions.java:1)
    at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:173)

I have read that some elements are generated on the client side and not seen in view source. But does that imply that we cannot access those elements at all which are not in View Source.

I have already tried waiting for the element to load for 30 seconds. But still I am getting the same error.

Please provide your inputs as I have googled a lot and not able to figure this out.

回答1:

The fact that the element you are looking for is not in the page source should not be a problem. The WebDriver API was introduced to explicitly add support for dynamic pages, e.g. where elements are added via JavaScript.

However it is not sufficient that an element is in the DOM, but the element also needs to be visible. WebDriver means to only allow interactions which also a user could do, so if an element is not visible, you should e.g. not be able to click on it via WebDriver.

来源：https://stackoverflow.com/questions/23949547/selenium-webdriver-not-able-to-find-elements-which-are-not-present-in-the-page-s

标签

java

selenium

selenium-webdriver

webdriver

htmlunit-driver