From a practical perspective (I have written dozens of "web-interactive" apps over the years), I finally settled on Watin combined with CSQuery.
Watin provide the basics of browser automation (interacting with buttons etc), while CSQuery lets you use jQuery style syntax to parse the page content.
I used Selenium for a while (also designed for automated testing of websites) with IE and FireFox, but found it to be prone to crashing when used for long-term scraping. All my current production systems are Watin + CSQuery and provide reliable scraping of multiple websites on a daily basis.
Note: I realise Watin has not been updated for a while, but "if it ain't broke, use it!" :)
