HtmlUnit + Selenium within Production

后端 未结 3 1697
长发绾君心
长发绾君心 2021-01-04 10:27

I am currently using HtmlUnit and Selenium to drive it (WebDriver) within my production code.

I am scaping and interacting with various websites programmatically wi

3条回答
  •  臣服心动
    2021-01-04 10:53

    WebDriver and Selenium are perfectly suited for production environnement. I use them quite extensively for 2 years now on a multi-machines/multi-datacenters distributed grid and had absolutely no performance nor stability problems we couldn't have coped with.

    Our preferred driver is the Firefox one (heavier than HTMLUnit, and harder to configure), and we had to tweak the grid to understand how many instances we can run. Our maximum for stability was 1 per core

    Our selenium/webdriver instances have run 24/7 for 2 years now (1 year with selenium 1, and the other migrating selenium 2/ WebDriver incrementally) and with an appropriate monitoring (you should monitor Memory Usage/CPU Usage) and a bunch of load testing, we had reached the good level where we have experienced several monthes without restarting a process

    We've used HTMLUnit extensively too, and are equally satisfied with this library

    The essential point of my post is : YES, these library are production-ready. But, as all production software, you'll have to benchmark their use to find the appropriate configuration for the optimal stability. I recommend you to use the Selenium Grid in production, which is a great way to parallelize process

提交回复
热议问题