Why doesn't asyncio always use executors?

前端 未结 1 1938
余生分开走
余生分开走 2020-12-18 05:45

I have to send a lot of HTTP requests, once all of them have returned, the program can continue. Sounds like a perfect match for asyncio. A bit naively, I wrapp

相关标签:
1条回答
  • 2020-12-18 06:01

    But as far as I can see, the requests module works perfectly - as long as you wrap it in an executor. Is there a reason to avoid wrapping something in an executor ?

    Running code in executor means to run it in OS threads.

    aiohttp and similar libraries allow to run non-blocking code without OS threads, using coroutines only.

    If you don't have much work, difference between OS threads and coroutines is not significant especially comparing to bottleneck - I/O operations. But once you have much work you can notice that OS threads perform relatively worse due to expensively context switching.

    For example, when I change your code to time.sleep(0.001) and range(100), my machine shows:

    asynchronous (executor) took 0.21461606299999997 seconds
    aiohttp took 0.12484742700000007 seconds
    

    And this difference will only increase according to number of requests.

    The purpose of asyncio is to speed up blocking io calls.

    Nope, purpose of asyncio is to provide convenient way to control execution flow. asyncio allows you to choose how flow works - based on coroutines and OS threads (when you use executor) or on pure coroutines (like aiohttp does).

    It's aiohttp's purpose to speed up things and it copes with the task as shown above :)

    0 讨论(0)
提交回复
热议问题