Python requests module multithreading

后端 未结 2 1944
旧时难觅i
旧时难觅i 2021-01-07 14:08

Is there a possible way to speed up my code using multiprocessing interface? The problem is that this interface uses map function, which works only with 1 function. But my c

2条回答
  •  小蘑菇
    小蘑菇 (楼主)
    2021-01-07 15:00

    Yes, you can use multiprocessing.

    from multiprocessing import Pool
    
    def f(line):
        session = requests.session()
        Login(session, line)
        Timer(session, line)
        Logout(session, line)        
    
    if __name__ == '__main__':
        urls = open('text.txt').read().splitlines()
        p = Pool(5)
        print(p.map(f, urls))
    

    The requests session cannot be global and shared between workers, every worker should use its own session.

    You write that you already "tried to combine my functions into one, but didn't get success". What exactly didn't work?

提交回复
热议问题