rate-limiting

aiohttp: rate limiting requests-per-second by domain

Deadly 提交于 2020-05-13 06:04:38
问题 I am writing a web crawler that is running parallel fetches for many different domains. I want to limit the number of requests-per-second that are made to each individual domain , but I do not care about the total number of connections that are open, or the total requests per second that are made across all domains. I want to maximize the number of open connections and requests-per-second overall, while limiting the number of requests-per-second made to individual domains. All of the

How can I rate limit network traffic on a docker container

故事扮演 提交于 2020-03-18 09:08:07
问题 I want to setup a docker container for a peer 2 peer app. This app doesn't have app level rate limiting so I'm attempting to set a limit at the container level. I would like to rate limit outgoing and incoming connections on all ports but the one used by the app's web UI. 回答1: I'm surprised at how difficult it was to find the answer to this question. Most answers on the various forums are incorrect (I tested them with two iperf3 nodes and found that the solutions didn't work or only limited

How can I rate limit network traffic on a docker container

我的未来我决定 提交于 2020-03-18 09:08:03
问题 I want to setup a docker container for a peer 2 peer app. This app doesn't have app level rate limiting so I'm attempting to set a limit at the container level. I would like to rate limit outgoing and incoming connections on all ports but the one used by the app's web UI. 回答1: I'm surprised at how difficult it was to find the answer to this question. Most answers on the various forums are incorrect (I tested them with two iperf3 nodes and found that the solutions didn't work or only limited

Should i use Sleep() or just deny them

二次信任 提交于 2020-02-29 21:37:31
问题 Im implementing a delay system so that any IP i deem abusive will automatically get an incremental delay via Sleep(). My question is, will this result in added CPU usage and thus kill my site anyways if the attacker just keeps opening new instances while being delayed? Or is the sleep() command use minimal CPU/memory and wont be much of a burden on a small script. I dont wish to flat out deny them as i'd rather they not know about the limit in an obvious way, but willing to hear why i should.

Rate limiting for Google/Firebase cloud functions?

感情迁移 提交于 2020-02-27 08:27:20
问题 I am using Firebase to develop an app that uses Cloud Functions as a REST API internally. My question is, is there an easy way to implement per-IP/per-user rate-limiting similar to what slack uses, except on a per-IP and per-user basis, rather than per-app (since it's all one app). Optional support for small bursts is preferable as well. Example code (see the // TODO: comments): exports.myCoolFunction = functions.https.onRequest((req, res) => { // TODO: implement IP rate-limiting here

How to change the rate limit whilst using a RateGate?

半城伤御伤魂 提交于 2020-02-04 03:52:27
问题 I am using the concise RateGate class to limit the number of requests I send to a server. My code looks something like this: var RateLimit = 35; using(var RateGate = new RateGate(RateLimit, TimeSpan.FromSeconds(1))) { for(var Run = 1; Run <= 50; Run++) { for(var Batch = 0; Batch < 200; Batch++) { // Do some work, then... MyClass MyClass; if(MyClass.RateLimitHit) { RateLimit--; } RateGate.WaitToProceed(); } } } Inside the if(MyClass.RateLimitHit) , I need to lower the rate limit by 1. Not just

Laravel API speed (too slow)

五迷三道 提交于 2020-01-25 20:27:25
问题 i'm trying to develop an API to store alarms coming from several applications into a database. I'm developing the api with Laravel. I've made a C program which makes multiple post requests to the API in order to see how many request can the laravel api process. I have the following route in api.php Route::post('/alarm', 'Api\v1\AlarmController@store'); In my controller i have made the store function which stores the alarm values received in the post request into the database function store

Facebook graph API response size limiting (error code 1)

与世无争的帅哥 提交于 2020-01-14 08:59:29
问题 Just sharing some information I came across while testing my application. Facebook Graph API implements rate limiting as described on their documentation page. Today I was trying to retrieve the feed from CNN facebook page and I got the following 500 error: {"error":{"code":1,"message":"Please reduce the amount of data you're asking for, then retry your request"}} This is the query I was trying to test: https://graph.facebook.com/v2.3/5550296508/feed?fields=id,actions,application,caption

Facebook graph API response size limiting (error code 1)

邮差的信 提交于 2020-01-14 08:59:09
问题 Just sharing some information I came across while testing my application. Facebook Graph API implements rate limiting as described on their documentation page. Today I was trying to retrieve the feed from CNN facebook page and I got the following 500 error: {"error":{"code":1,"message":"Please reduce the amount of data you're asking for, then retry your request"}} This is the query I was trying to test: https://graph.facebook.com/v2.3/5550296508/feed?fields=id,actions,application,caption

Rate limit in nginx based on http header

二次信任 提交于 2020-01-14 07:24:48
问题 Maybe I am asking a poor question but I want to apply rate limit in nginx based on custom http header rather than IP based. My IP based configuration is working but I am not able to get around using custom http header. What I want is that if a particular header is present in http request then rate limiting should be applied otherwise not. conf file http { limit_req_zone $http_userAndroidId zone=one:10m rate=1r/s; location ^~ /mobileapp{ set $no_cache 1; # set rate limit by pulkit limit_req