eventlet

Eventlet timeout not exiting

由 拥有回忆 提交于 2019-12-11 10:41:50
问题 Why does eventlet allow it to finish the 6 seconds, when it should exit the indentation after 5 seconds? >>> with eventlet.Timeout(5): time.sleep(6) x = 1 >>> x 1 回答1: Eventlet provides cooperative multithreading. Which means you need to yield control to give hub or coroutines (in this case, hub implements timeouts) chance to run. To yield control: either use green versions of IO and sleep or execute eventlet.monkey_patch() , now you can use regular time , socket , etc modules, replaced by

Django exceeds maximum Postgres connections

由 自闭症网瘾萝莉.ら 提交于 2019-12-07 08:01:16
问题 I am experiencing a problem with a Django application that is exceeding the maximum number of simultaneous connections (100) to Postgres when running through Gunicorn with async eventlet workers. When the connection limit it reached the application starts returning 500 -errors until new connections can be established. This is my database configuration: DATABASES = { 'default': { 'ENGINE': 'django.contrib.gis.db.backends.postgis', 'NAME': 'django', 'USER': 'django', 'HOST': 'postgres', 'PORT':

How to make non-blocking raw_input when using eventlet.monkey_patch() and why it block everything, even when executed on another thread?

由 强颜欢笑 提交于 2019-12-06 11:24:50
问题 I wrote this minimum code to explain my case: import threading import time import eventlet eventlet.monkey_patch() def printing_function(): while True: # here i want to do some work print "printing" time.sleep(1) if __name__ == '__main__': thread = threading.Thread(target=printing_function) thread.start() while True: # here i want to wait for users input raw_input("raw input\n") print "inside main loop" time.sleep(1) Even i have 2 threads, both of them are blocked when i call raw_input. When

Unterstanding eventlet.wsgi.server

由 做~自己de王妃 提交于 2019-12-06 05:10:51
I have this simple Python programm: from eventlet import wsgi import eventlet from eventlet.green import time def hello_world(env, start_response): print "got request" time.sleep(10) start_response('200 OK', [('Content-Type', 'text/plain')]) return ['Hello, World!\n'] wsgi.server(eventlet.listen(('', 8090)), hello_world) So when i run it, and open http://localhost:8090/ on my browser multiple times, got request is only printed after the first request was already processed (after 10 seconds). It seems like eventlet.wsgi.server is processing the requests synchronously. But I am using the "green"

Django exceeds maximum Postgres connections

由 梦想的初衷 提交于 2019-12-05 15:59:56
I am experiencing a problem with a Django application that is exceeding the maximum number of simultaneous connections (100) to Postgres when running through Gunicorn with async eventlet workers. When the connection limit it reached the application starts returning 500 -errors until new connections can be established. This is my database configuration: DATABASES = { 'default': { 'ENGINE': 'django.contrib.gis.db.backends.postgis', 'NAME': 'django', 'USER': 'django', 'HOST': 'postgres', 'PORT': 5432, 'CONN_MAX_AGE': 60, } } This is how Gunicorn is started: gunicorn --bind 0.0.0.0:8080 --worker

What are the advantages of multithreaded programming in Python?

由 我怕爱的太早我们不能终老 提交于 2019-12-05 02:07:34
问题 When I hear about multithreaded programming, I think about the opportunity to accelerate my program, but it is not? import eventlet from eventlet.green import socket from iptools import IpRangeList class Scanner(object): def __init__(self, ip_range, port_range, workers_num): self.workers_num = workers_num or 1000 self.ip_range = self._get_ip_range(ip_range) self.port_range = self._get_port_range(port_range) self.scaned_range = self._get_scaned_range() def _get_ip_range(self, ip_range): return

How to make non-blocking raw_input when using eventlet.monkey_patch() and why it block everything, even when executed on another thread?

由 て烟熏妆下的殇ゞ 提交于 2019-12-04 17:22:59
I wrote this minimum code to explain my case: import threading import time import eventlet eventlet.monkey_patch() def printing_function(): while True: # here i want to do some work print "printing" time.sleep(1) if __name__ == '__main__': thread = threading.Thread(target=printing_function) thread.start() while True: # here i want to wait for users input raw_input("raw input\n") print "inside main loop" time.sleep(1) Even i have 2 threads, both of them are blocked when i call raw_input. When i comment out eventlet.monkey_patch(), only one thread is blocked and another keep printing "printing".

How to efficiently do many tasks a “little later” in Python?

由 青春壹個敷衍的年華 提交于 2019-12-03 03:13:25
问题 I have a process, that needs to perform a bunch of actions "later" (after 10-60 seconds usually). The problem is that those "later" actions can be a lot (1000s), so using a Thread per task is not viable. I know for the existence of tools like gevent and eventlet, but one of the problem is that the process uses zeromq for communication so I would need some integration (eventlet already has it). What I'm wondering is What are my options? So, suggestions are welcome, in the lines of libraries

Fastest way to download 3 million objects from a S3 bucket

由 允我心安 提交于 2019-12-02 15:35:38
I've tried using Python + boto + multiprocessing, S3cmd and J3tset but struggling with all of them. Any suggestions, perhaps a ready-made script you've been using or another way I don't know of? EDIT: eventlet+boto is a worthwhile solution as mentioned below. Found a good eventlet reference article here http://web.archive.org/web/20110520140439/http://teddziuba.com/2010/02/eventlet-asynchronous-io-for-g.html I've added the python script that I'm using right now below. Jagtesh Chadha Okay, I figured out a solution based on @Matt Billenstien's hint. It uses eventlet library. The first step is

How to combine multiprocessing and eventlet

由 岁酱吖の 提交于 2019-12-02 05:37:41
问题 I have a task need to start 2 processes and within each process need to start 2 threads to do really work. Below is the source code I used to simulate my use case. import multiprocessing import eventlet def subworker(num1, num2): print 'Start subworker %d,%d\n' % (num1, num2) eventlet.sleep(10) print 'End subworker %d,%d\n' % (num1, num2) def worker(**kwargs): number = kwargs['number'] pool = eventlet.GreenPool(size=2) pool.spawn_n(subworker, number, 1) pool.spawn_n(subworker, number, 2) pool
订阅 eventlet