Behavior of the scrapy xpath selector on h1-h6 tags

Question

Why does the following two code snippets give different outputs? The only difference between them is that the h1 tag in the first case is replaced with an

Answer 1

Including p tags inside h# is invalid according to W3C. You can see more about this here

Anyway, to bypass this and just work with any xml structure you can just change the type like this:

sel = Selector(text="anyxml", type="xml")

This will respect any xml structure.