I am using urllib2.urlopen() and my process is getting blocked
I am aware that urllib2.urlopen() has default timeout.
How to make the call unblockable?
The backtrace is
(gdb) bt
#0 0x0000003c6200dc35 in recv () from /lib64/libpthread.so.0
#1 0x00002b88add08137 in ?? () from /usr/lib64/python2.6/lib-dynload/_socketmodule.so
#2 0x00002b88add0830e in ?? () from /usr/lib64/python2.6/lib-dynload/_socketmodule.so
#3 0x000000310b2d8e19 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0
If your problem is that you need to urllib to finish reading
read() operation is blocking operation in Python.
If you want to create asynchronous requests
Do reading in non-main thread http://docs.python.org/library/threading.html
Use
requestslibrary and asynchronous requests http://docs.python-requests.org/en/latest/user/advanced/#asynchronous-requests
If your problem is need to set timeout
Again, use requests library as mentioned above.
You can try using strace (or similar) tool to figure out what the actual system call is that is blocking your python script, e.g on linux: $ strace python yourscript.py
yourscript.py:
from urllib2 import urlopen
urlopen("http://somesite.local/foobar.html")
$ strace python yourscript.py
... lots of system call stripped ...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("127.0.0.1")}, 16
来源：https://stackoverflow.com/questions/11664185/python-urllib2-urlopenurl-process-block