I am currently using HtmlUnit and Selenium to drive it (WebDriver) within my production code.
I am scaping and interacting with various websites programmatically wi
WebDriver and Selenium are perfectly suited for production environnement. I use them quite extensively for 2 years now on a multi-machines/multi-datacenters distributed grid and had absolutely no performance nor stability problems we couldn't have coped with.
Our preferred driver is the Firefox one (heavier than HTMLUnit, and harder to configure), and we had to tweak the grid to understand how many instances we can run. Our maximum for stability was 1 per core
Our selenium/webdriver instances have run 24/7 for 2 years now (1 year with selenium 1, and the other migrating selenium 2/ WebDriver incrementally) and with an appropriate monitoring (you should monitor Memory Usage/CPU Usage) and a bunch of load testing, we had reached the good level where we have experienced several monthes without restarting a process
We've used HTMLUnit extensively too, and are equally satisfied with this library
The essential point of my post is : YES, these library are production-ready. But, as all production software, you'll have to benchmark their use to find the appropriate configuration for the optimal stability. I recommend you to use the Selenium Grid in production, which is a great way to parallelize process