Whats the difference between getting text
and innerHTML
when using selenium. Even though we have text under particular element, when we perform
Chrome (i'm not sure about other browsers) ignores the extra spaces within the HTML code and displays as a single space.
<div><span>Example Text</span></div> # notice the two spaces
.get_attribute('innerHTML')
will return the double-spaced text, which is what you would see when you inspect element), while .text
will return the string with only 1 space.
>>> print(element.get_attribute('innerHTML'))
'Example Text'
>>> print(element.text)
'Example Text'
This difference is not trivial as the following will result in a NoSuchElementException.
>>> arg = '//div[contains(text(),"Example Text")]'
>>> driver.find_element_by_xpath(arg)
Similarly, .get_attribute('innerHTML')
for the following returns Example Text
, while .text
returns Example Text
.
<div><span>Example Text</span></div>
To start with, text
is a property where as innerHTML
is an attribute. Fundamentally there are some differences between a property and an attribute.
get_attribute(innerHTML) gets the innerHTML
of the element.
This method will first try to return the value of a property with the given name. If a property with that name doesn’t exist, it returns the value of the attribute
with the same name. If there’s no attribute
with that name, None
is returned.
Values which are considered truthy, that is equals true
or false
, are returned as booleans. All other non-None
values are returned as strings. For attributes or properties which do not exist, None
is returned.
Args:
innerHTML - Name of the attribute/property to retrieve.
Example:
# Extract the text of an element.
my_text = target_element.get_attribute("innerHTML")
text gets the text of the element.
Definition:
def text(self):
"""The text of the element."""
return self._execute(Command.GET_ELEMENT_TEXT)['value']
Example:
# Extract the text of an element.
my_text = target_element.text
Still sounds similar? Read below ...
When the browser loads the page, it parses the HTML and generates DOM objects from it. For element nodes, most standard HTML attributes automatically become properties of DOM objects.
For instance, if the tag is:
<body id="page">
then the DOM object has body.id="page"
.
Note: The attribute-property mapping is not one-to-one!
In HTML, tags may have attributes. When the browser parses the HTML to create DOM objects for tags, it recognizes standard attributes and creates DOM properties from them.
So when an element has id or another standard attribute, the corresponding property gets created. But that doesn’t happen if the attribute is non-standard.
Note: A standard attribute for one element can be unknown for another one. For instance,
type
is standard attribute for <input> tag, but not for <body> tag. Standard attributes are described in the specification for the corresponding element class.
So, if an attribute is non-standard, there won’t be a DOM-property for it. In that case all attributes are accessible by using the following methods:
elem.hasAttribute(name)
: checks for existence.elem.getAttribute(name)
: gets the value.elem.setAttribute(name, value)
: sets the value.elem.removeAttribute(name)
: removes the attribute.An example of reading a non-standard property:
<body something="non-standard">
<script>
alert(document.body.getAttribute('something')); // non-standard
</script>
</body>
When a standard attribute changes, the corresponding property is auto-updated, and (with some exceptions) vice versa. But there are exclusions, for instance input.value
synchronizes only from attribute
-> to property
, but not back. This feature actually comes in handy, because the user may modify value, and then after it, if we want to recover the "original" value from HTML, it’s in the attribute.
As per Attributes and Properties in python when we reference an attribute of an object with something like someObject.someAttr
, Python uses several special methods to get the someAttr
attribute of the object. In the simplest case, attributes are simply instance variables.
In a broader perspective:
someObj.name
.__dict__
of an object.someObj.name
, the default behavior is effectively someObj.__dict__['name']
In Python we can bind getter
, setter
(and deleter
) functions with an attribute name, using the built-in property()
function or @property
decorator. When we do this, each reference to an attribute has the syntax of direct access to an instance variable, but it invokes the given method function.
I have just selected the css selector and used below code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://www.costco.com/Weatherproof%C2%AE-Men's-Ultra-Tech-Jacket.product.100106552.html")
print driver.find_element_by_css_selector(".product-h1-container.visible-xl-block>h1").text
and it prints:
Weatherproof® Men's Ultra Tech Jacket
The problem is h1[itemprop='name']
selector on chrome or firefox are returning 2 matching nodes while .product-h1-container.visible-xl-block>h1
is returning only one matching node thats why its prining what is expected
To prove my point run below code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://www.costco.com/Weatherproof%C2%AE-Men's-Ultra-Tech-Jacket.product.100106552.html")
x= driver.find_elements_by_css_selector("h1[itemprop='name'] ")
for i in x:
print "This is line " , i.text
It will print
This is line
This is line Weatherproof® Men's Ultra Tech Jacket
Because select_element_by_css_selector selects the first element with matching selector and that does not contain any text so it does not print. Hope you understand now
For instance, <div><span>Example Text</span></div>
.get_attribute("innerHTML")
gives you the actual HTML inside the current element. So theDivElement.get_attribute("innerHTML")
returns "<span>Example Text</span>
"
.text
gives you only text, not include HTML node. So theDivElement.text
returns "Example Text
"
Please note that the algorithm for .text
depends on webdriver of each browser. In some cases such as element is hidden, you might get different text when you use different webdriver.
I usually get text from .get_attribute("innerText")
instead of .text
so I can handle the all the case.
.text will retrieve an empty string of the text in not present in the view port, so you can sroll the object into the viewport and try .text it should retrive the value.
On the contrary innerhtml can get the value even of it is present out side the view port