urllib

网络请求模块 urllib(标准模块）python操作网络，也就是打开一个网站，或者请求一个http接口，使用urllib模块 用到模块中 request parse 两个方法 由于urllib模块，传参麻烦，需要把字典类型parse下变为k=v,post还要把参数转为二进制；接收的值是bytes类型，还要解码，变为字符串，同时直接用的话还要转为为json。比较麻烦。 更好的就是用 第三方模块 request 模块 　　urllib模块 的 get请求 ----------------------------------------------- urllib模块 post请求 --------------------------------------------------------------------------------------------------------------------------------------------- *（requests模块） 　　 get()请求 requests.get(url,data) post请求 requests.post(url,data) ----------------- post 请求 入参是 json类型的 可以通过在post请求中指定参数类型 requests.post(url,json=data)

How do I check to see if urllib.urlretrieve(url, file_name) has completed before allowing my program to advance to the next statement? Take for example the following code snippet: import traceback import sys import Image from urllib import urlretrieve try: print "Downloading gif....." urlretrieve(imgUrl, "tides.gif") # Allow time for image to download/save: time.sleep(5) print "Gif Downloaded." except: print "Failed to Download new GIF" raw_input('Press Enter to exit...') sys.exit() try: print "Converting GIF to JPG...." Image.open("tides.gif").convert('RGB').save("tides.jpg") print "Image

Sorry that the title wasn't very clear, basically I have a list with a whole series of url's, with the intention of downloading the ones that are pictures. Is there anyway to check if the webpage is an image, so that I can just skip over the ones that arent? Thanks in advance You can use requests module. Make a head request and check the content type. Head request will not download the response body. import requests response = requests.head(url) print response.headers.get('content-type') jfs There is no reliable way. But you could find a solution that might be "good enough" in your case. You

I want to get the Content-Length value from the meta variable. I need to get the size of the file that I want to download. But the last line returns an error, HTTPMessage object has no attribute getheaders . import urllib.request import http.client #----HTTP HANDLING PART---- url = "http://client.akamai.com/install/test-objects/10MB.bin" file_name = url.split('/')[-1] d = urllib.request.urlopen(url) f = open(file_name, 'wb') #----GET FILE SIZE---- meta = d.info() print ("Download Details", meta) file_size = int(meta.getheaders("Content-Length")[0]) It looks like you are using Python 3, and

I have the following simple code: import urllib2 import sys sys.path.append('../BeautifulSoup/BeautifulSoup-3.1.0.1') from BeautifulSoup import * page='http://en.wikipedia.org/wiki/Main_Page' c=urllib2.urlopen(page) This code generates the following error messages: c=urllib2.urlopen(page) File "/usr/lib64/python2.4/urllib2.py", line 130, in urlopen return _opener.open(url, data) File "/usr/lib64/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib64/python2.4/urllib2.py", line 471, in http_response response = self.parent.error( File "/usr/lib64/python2.4

I have a simple website I'm testing. It's running on localhost and I can access it in my web browser. The index page is simply the word "running". urllib.urlopen will successfully read the page but urllib2.urlopen will not. Here's a script which demonstrates the problem (this is the actual script and not a simplification of a different test script): import urllib, urllib2 print urllib.urlopen("http://127.0.0.1").read() # prints "running" print urllib2.urlopen("http://127.0.0.1").read() # throws an exception Here's the stack trace: Traceback (most recent call last): File "urltest.py", line 5,

I'm trying to get a response from urllib and decode it to a readable format. The text is in Hebrew and also contains characters like { and / top page coding is: # -*- coding: utf-8 -*- raw string is: b'\xff\xfe{\x00 \x00\r\x00\n\x00"\x00i\x00d\x00"\x00 \x00:\x00 \x00"\x001\x004\x000\x004\x008\x003\x000\x000\x006\x004\x006\x009\x006\x00"\x00,\x00\r\x00\n\x00"\x00t\x00i\x00t\x00l\x00e\x00"\x00 \x00:\x00 \x00"\x00\xe4\x05\xd9\x05\xe7\x05\xd5\x05\xd3\x05 \x00\xd4\x05\xe2\x05\xd5\x05\xe8\x05\xe3\x05 \x00\xd4\x05\xea\x05\xe8\x05\xe2\x05\xd4\x05 \x00\xd1\x05\xde\x05\xe8\x05\xd7\x05\xd1\x05 \x00"\x00,