Whats the best screen scraping language? [closed]

Answer 1

HTML Agility Pack (c#)

1. XPath is borked, the way the html is cleaned to make it xml compliant it will drop tags and you have to adjust the expression to get it to work.
2. simple to use

Mozilla Parser (Java)

1. Solid XPath support

2. you have to set enviroment variables before it will work which is a pain

3. casting between org.dom4j.Node and org.w3c.dom.Node to get different properties is a real pain

4. dies on non-standard html (0.3 fixes this)

5. best solution for XPath

6. problems accessing data on Nodes in a NodeList

   use a for(int i=1;i<=list_size;i++) to get around that

Beautiful Soup (Python)

I don't have much experience but here's what I've found

1. no XPath support
2. nice interface to pathing html

---

I prefer Mozilla HTML Parser