Get the inner HTML of a element in lxml

Question

I am trying to get the HTML content of child node with lxml and xpath in Python. As shown in code below, I want to find the html content of the each of product nodes. Does i

Answer 1

After right clicking (copy, copy xpath) on the specific field you want (in chrome's inspector), you might get something like this:

//*[@id="specialID"]/div[12]/div[2]/h4/text()[1]

If you wanted that text element for each "specialID"

//*[@id="specialID"]/div/div[2]/h4/text()[1]

You could select another field and it'll interleave the results

//*[@id="specialID"]/div/div[2]/h4/text()[1] | //*[@id="specialID"]/div/some/weird/path[95]

Example could be improved, but it illustrates the point:

//*[@id="mw-content-text"]/div/ul[1]/li[11]/text()

from lxml import html
import requests
page = requests.get('https://en.wikipedia.org/wiki/Web_scraping')
tree = html.fromstring(page.content)
data = tree.xpath('//*[@id="mw-content-text"]/div/ul[1]/li/a/text() | //*[@id="mw-content-text"]/div/ul[1]/li/text()[1]')
print(len(data))
for i in range(len(data)):
    print(data[i])