urllib

How can I create a percent encoded URL from complete URL?

孤人 提交于 2019-12-12 03:37:04
问题 The input url is mixed ascii and multi byte code. And I can't change this strings. I would like to download this url, but an error occured ordinal not in range(128) . input_url = "http://sample.jp/api?v1=aaa&v2=日本語&v3=ccc" req = urllib.request.Request(input_url) resp = None try: resp = urllib.request.urlopen(req) except UnicodeEncodeError as e: print(e.reason) # I had an error `ordinal not in range(128)` return resp First, I tried urllib.parse.quote() but the result is http%3a%2f%2fsample

Connect to Microsoft Azure Machine Learning Studio Api with ruby instead of python use net/http gem instead of urllib2

断了今生、忘了曾经 提交于 2019-12-12 02:39:05
问题 I need to connect to the Microsoft Azure Machine Learning Studio Api with ruby instead of python. Can someone help me translate this python code into ruby using the net/http gem. import urllib2 # If you are using Python 3+, import urllib instead of urllib2 import json data = { "Id": "score00001", "Instance": { "FeatureVector": { "value_1"= "1", "value_2"= "2", "value_3"= "3", "value_4"= "4", "value_5"= "5", "value_6"= "6", "value_7"= "7", "value_8"= "8", "value_9"= "9", "value_10"= "10", },

Getting the options in a http request status 300

泪湿孤枕 提交于 2019-12-12 02:09:16
问题 I read that when I get this error I should specify better the url. I assume that I should specify between two displayed or accessible options. How can I do that? In urllib or its tutorial I couldn't find anything. My assumption is true? Can I read somewhere the possible url? When I open this url in my browser I am redirected to a new url. The url I try to access: http://www.uniprot.org/uniprot/P08198_CSG_HALHA.fasta The new url I am redirected: http://www.uniprot.org/uniprot/?query=replaces

retrieving essential data from a webpage using python

血红的双手。 提交于 2019-12-11 23:39:50
问题 Following is a part of a webpage i downloaded with urlretrieve (urllib). I want to write only this data from the webpage given below in to another text file as: ENGINEERING MATHEMATICS-IV, 4 ,36 ,40 , F ENVIRONMENTAL STUDIES, 47, 36, 83 , p ... .. . likeways.. which module should i use?, which all commands?? thanx in advance..:) part of the webpage <td>ENGINEERING MATHEMATICS-IV</td> <td align=center>4</td> <td align=center>36</td> <td align=center>40</td> <td align=center>F</td> </tr> <tr

How do I fix this IOError: [Errno socket error] [Errno 11004]?

喜欢而已 提交于 2019-12-11 18:42:45
问题 This simple Python 3 script: import urllib.request host = "scholar.google.com" link = "/scholar.bib?q=info:K7uZdMSvdQ0J:scholar.google.com/&output=citation&hl=en&as_sdt=1,14&ct=citation&cd=0" url = "http://" + host + link filename = "cite0.bib" print(url) urllib.request.urlretrieve("http://scholar.google.com" + url, filename) raises this exception: Traceback (most recent call last): File "C:/Users/ricardo/Desktop/Google-Scholar/BibTex/test2.py", line 8, in <module> urllib.request.urlretrieve(

How do I control Selenium PDF and Excel files download behavior?

可紊 提交于 2019-12-11 17:53:37
问题 I want to download all the tender documents from this url 'http://www.ha.org.hk/haho/ho/bssd/T18G014Pc.htm' I'm using selenium to go through each tender links and download the files. However, my scraper couldn't handle the Excel download behavior. Currently, it handles PDF files pretty well. from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from

Webscraping with urllib

我的未来我决定 提交于 2019-12-11 17:51:59
问题 I am looking to get some information off the CME website Namely I want to get the Futures Yield and the Futures DV01 for the 10y Treasury Note Future. Found this little snippet on an old thread: import urllib.request class AppURLopener(urllib.request.FancyURLopener): version = "Mozilla/5.0" opener = AppURLopener() fh = opener.open('http://www.cmegroup.com/tools-information/quikstrike/treasury-analytics.html') It throws a deprecation warning and I am not quite sure how I get the info from the

Django request XML file with SSL IO error

好久不见. 提交于 2019-12-11 14:51:49
问题 I'm making a Django website and am fairly new. In this webapp I need to use this API which will spit out an xml file with the requested data from the database. Basically the API URL is: https://adminuser:password@url.to.database.com/database.getdata?arg=1&arg2=0 So in my python views.py I have: def fetch_xml(url): import urllib import xml.etree.cElementTree as xml_parser u = urllib.URLopener(None) usock = u.open(url) rawdata = usock.read() usock.close() return xml_parser.fromstring(rawdata)

Python 2.7 error downloading Dropbox file by urllib.urlretrieve

笑着哭i 提交于 2019-12-11 13:54:33
问题 The code I used (don't mind the image, it's a test one): import urllib urllib.urlretrieve('https%3A//dl.dropboxusercontent.com/u/95587456/Evenimente/1.jpg','/storage/emulated/0/Temp/1.jpg') I tried to download an image by its public dropbox link, only for this error to pop up: I/python ( 3750): Traceback (most recent call last): I/python ( 3750): File "/home/cristi/Desktop/AplicatieMinister/.buildozer/android/app/main.py", line 102, in <module> "/home/cristi/Desktop/AplicatieMinister/

How to download a image in flickr by Python urllib.urlretrieve()?

若如初见. 提交于 2019-12-11 13:01:57
问题 I have a problem that when dowloading a image from flickr.com,the python function urllib.urlretrieve() always return an error [Errno socket error] (10060, 'Operation timed out') for example: import urllib url = "http://farm3.static.flickr.com/2659/4207030547_23e6000d29_o.gif" urllib.urlretrieve(url,"C://tmp.gif") I am Chinese,and I dont know if the "time out" has anything to do with the speed of the the internet in China. Now it falied in downing the .gif! what should i do about this? THX~~~