问题
import asyncio
import httpx
from datetime import datetime
async def request_test(url):
async with httpx.AsyncClient() as client:
r = await client.get(url, timeout=None, headers=None)
return len(r.text)
async def main(rest_api_url_list ):
futures = [asyncio.ensure_future(request_test(url)) for url in rest_api_url_list ]
results = await asyncio.gather(*futures)
print(results)
print(len(results))
start = datetime.now()
rest_api_url_list = [~~~~~~~~~~~~~] # 2000EA
loop = asyncio.get_event_loop()
loop.run_until_complete(main(rest_api_url_list ))
loop.close()
end = datetime.now()
Hi, I have 2000 api adress. And I need to call 2000 concurrently in one VM. So, I used the asyncio library to modify the code as above. But, This solution is not satisfactory. How can I increase the effect of parallel processing? I think I have to use multiprocessing and asyncio at the same time.
来源:https://stackoverflow.com/questions/64851881/how-to-use-both-python-multiprocessing-and-asyncio