Regulating / rate limiting ruby mechanize

问题

I need to regulate how often a Mechanize instance connects with an API (once every 2 seconds, so limit connections to that or more)

So this:

instance.pre_connect_hooks << Proc.new { sleep 2 }

I had thought this would work, and it sort of does BUT now every method in that class sleeps for 2 seconds, as if the mechanize instance is touched and told to hold 2 seconds. I'm going to try a post connect hook, but it is obvious I need something a bit more elaborate, but what I don't know what at this point.

Code is more explanation so if you are interested following along: https://github.com/blueblank/reddit_modbot, otherwise my question concerns how to efficiently and effectively rate limit a Mechanize instance to within a specific time frame specified by an API (where overstepping that limit results in dropped requests and bans). Also, I'm guessing I need to better integrate a mechanize instance to my class as well, any pointers on that appreciated as well.

回答1:

Pre and post connect hooks are called on every connect, so if there is some redirection it could trigger many times for one request. Try history_added which only gets called once:

instance.history_added = Proc.new {sleep 2}

回答2:

I use SlowWeb to rate limit calls to a specific URL.

require 'slowweb'
SlowWeb.limit('example.com', 10, 60)

In this case calls to example.com domain are limited to 10 requests every 60 seconds.

来源：https://stackoverflow.com/questions/9241625/regulating-rate-limiting-ruby-mechanize