urllib

Write contents of URL request to file

由 大憨熊 提交于 2019-12-07 16:38:26
问题 I am trying to fetch a list from a php file using python and save it to a file: import urllib.request page = urllib.request.urlopen('http://crypto-bot.hopto.org/server/list.php') f = open("test.txt", "w") f.write(str(page)) f.close() print(page.read()) Output on screen (divided onto four lines for readability): ALF\nAMC\nANC\nARG\nBQC\nBTB\nBTE\nBTG\nBUK\nCAP\nCGB\nCLR\nCMC\nCRC\nCSC\nDGC\n DMD\nELC\nEMD\nFRC\nFRK\nFST\nFTC\nGDC\nGLC\nGLD\nGLX\nHBN\nIXC\nKGC\nLBW\nLKY\n LTC\nMEC\nMNC\nNBL

Python requests error 10060

由 给你一囗甜甜゛ 提交于 2019-12-07 15:32:32
I have a script that crawls a website. Untill today it ran perfect, However it does not do so now. it give sme the following error: Connection Aborted Error(10060 ' A connection attempt failed becvause the connected party did not properly respond after a period of time, or established a connection failed because connected host has failed to respond' I have been looking into answers ans settings but i cannot figure out how to fix this... In IE i am not using any Proxy (Connection -> Lan Settings-> Proxy = Disabled) it breaks in this piece of code, somethimes the first run, somethimes the 2nd..

python urllib error - AttributeError: 'module' object has no attribute 'request'

由 |▌冷眼眸甩不掉的悲伤 提交于 2019-12-07 12:19:32
问题 I am trying out a tutorial code which fetches the html code form a website and prints it. I'm using python 3.4.0 on ubuntu. The code: import urllib.request page = urllib.request.urlopen("http://www.brainjar.com/java/host/test.html") text = page.read().decode("utf8") print(text) I saw previous solutions and tried them, I also tried importing only urllib but it still doesn't work. The error message displayed is as shown: Traceback (most recent call last): File "string.py", line 1, in <module>

unbuffered urllib2.urlopen

由 匆匆过客 提交于 2019-12-07 11:17:29
I have client for web interface to long running process. I'd like to have output from that process to be displayed as it comes. Works great with urllib.urlopen() , but it doesn't have timeout parameter. On the other hand with urllib2.urlopen() the output is buffered. Is there a easy way to disable that buffer? A quick hack that has occurred to me is to use urllib.urlopen() with threading.Timer() to emulate timeout. But that's only quick and dirty hack. urllib2 is buffered when you just call read() you could define a size to read and therefore disable buffering. for example: import urllib2

bitfinex api v2 error, invalid key

由 痞子三分冷 提交于 2019-12-07 11:07:26
问题 I am trying to make a basic authenticated api call to their new v2 api and getting an invalid api key error returned. I reissued the api key just to verify, same error. from time import time import urllib.request import urllib.parse import hashlib import hmac APIkey = b'myapikeyyouarenotsupposedtosee' secret = b'myceeeeecretkeyyyy' url = 'https://api.bitfinex.com/v2/auth/r/wallets' payload = { #'request':'/auth/r/wallets', 'nonce': int(time() * 1000), } paybytes = urllib.parse.urlencode

Fetch first n bytes from the URL

由 孤者浪人 提交于 2019-12-07 08:21:48
问题 Is that possible to fetch only a number of bytes from some URL and then close the connection with urllib/urllib2? Or even may be a part from n-th byte to k-th? There is a page on that side and I don't need to load the whole page, only a piece of it. 回答1: You can set the Range header to request a certain range of bytes, but you are dependent on the server to honor the request: import urllib2 req = urllib2.Request('http://www.python.org/') # # Here we request that bytes 18000--19000 be

Converting Unicode objects with non-ASCII symbols in them into strings objects (in Python)

由 こ雲淡風輕ζ 提交于 2019-12-07 07:05:18
问题 I want to send Chinese characters to be translated by an online service, and have the resulting English string returned. I'm using simple JSON and urllib for this. And yes, I am declaring. # -*- coding: utf-8 -*- on top of my code. Now everything works fine if I feed urllib a string type object, even if that object contains what would be Unicode information. My function is called translate . For example: stringtest1 = '無與倫比的美麗' print translate(stringtest1) results in the proper translation

Change python byte type to string

由 自古美人都是妖i 提交于 2019-12-07 05:59:23
问题 I'm using python to play with the stackoverflow API. I run the following commands: f = urllib.request.urlopen('http://api.stackoverflow.com/1.0/stats') d = f.read() The type of d is class 'bytes' and if I print it it looks like: b'\x1f\x8b\x08\x00\x00\x00 .... etc I tried d=f.read().decode('utf-8') as that is the charset indicated in the header, but I get a 'utf8' codec can't decode byte 0x8b in position 1" error message How do I convert the byte object I received from my urllib.request call

Open Windows shared folder through linux machine

由 爷，独闯天下 提交于 2019-12-07 05:43:48
问题 I am using python 2.5 on Ubuntu, and there's a machine in the same network called machine1 . The folder is shared. How to to get a file in a specific folder of that machine? I have tried, with no success: urllib.urlopen('\\machine1\folder\file.txt') 回答1: Linux has a utiliy called smbmount , which can be found in package smbutils I believe. This is a command line utility which mounts a Windows share to a directory on the local machine, optionally with username/password. smbmount is I believe a

Python, Detect is a URL needs to be HTTPS vs HTTP

由 偶尔善良 提交于 2019-12-07 02:07:41
问题 Using the python standard library, is there a way to determine if a given web address should use HTTP or HTTPS? If you hit a site using HTTP://.com is there a standard error code that says hey dummy it should be 'HTTPS' not http? Thank you 回答1: Did u make any sort of testing? The short, prematural answer of your questions is: Does not exist should use... it's your preference, or a server decision at all, because of redirects. Some servers does allow only https, and when you call http does
订阅 urllib