Throttling requests by IP address on Apache? [closed]

Answer 1

To avoid dos attacks/web scraping you can explore mod_evasive which provides various configuration to block requests. http://www.zdziarski.com/blog/?page_id=442

It can be useful for basic protection, however, it won't be sufficient against a determined and experienced attacker, who can attack from an internal network or use an array of server proxies to hide his IP.

Answer 2

"Is this the right way ... at the web server level?" It's probably the best option you have. It might be good to have different thresholds on different parts of your site: you may be more willing to throttle certain kinds of traffic than others. But ideally these kinds of settings would be managed at the network level.

"What's a good limit ... ?" It completely depends on your traffic. How much you expect, where your real users come from, etc.

How to do it? It is possible to write rules to handle this sort of thing in ModSecurity, which also defends against some other stuff. As with the mod_evasive answer, this won't fully protect you against attackers with a lot of resources at their disposal, but it would force them to step up their game.

I don't think there's anything "built into" Apache httpd that will facilitate this. The expectation would be that issues with an abusive IP address (i.e., network traffic issues) are managed at the network level.

EDIT:

Since you comment elsewhere that you are using Rackspace for hosting, you might want to check out their load balancer API.