Why the amount of greenlets will impact the elapsed time of the responses

独自空忆成欢 提交于 2019-12-24 18:53:22

问题


I'm using Python coroutine library gevent and monkey patch to increase the concurrency of http requests. But I noticed the elapsed time of the responses increased while the concurrency increased. Below the sample code:

import gevent
from gevent import monkey
import requests

monkey.patch_all(thread=False)


def action():
    resp = requests.get("https://www.google.com")
    if resp.status_code == 200:
        print resp.elapsed.total_seconds()


jobs = []
for i in range(100):
    jobs.append(gevent.spawn(action))

gevent.joinall(jobs)

When 10 greenlets were spawned, the elapsed time was around 0.9 seconds, but when the greenlets number was increased to 100, the elapsed time was around 1.6 ~ 2.0 seconds. Why this happened?


回答1:


greenlets are still single threaded, meaning they can only do one thing at a time. For any process that is cpu intensive this will incur a delay. Yes it is asynchronous, but it is not multiprocessed, so if something uses 1 second of CPU, you have delayed the results of any subsequent greenlet by 1 second.

So as your queue grows, the delay even if just ms, becomes noticeable.



来源:https://stackoverflow.com/questions/50209009/why-the-amount-of-greenlets-will-impact-the-elapsed-time-of-the-responses

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!