Use instead of + for space in python query parameters

Question

I have written the following python script, using python requests (http://requests.readthedocs.org/en/latest/):

import requests

payload = {\'key1\': \'value  1\         


        

Answer 1

from urllib.parse import urlencode

def to_query_string(params):
    return urlencode(params, doseq=True).replace('+', '%20')

---

You could pass a string to params instead of a dictionary, and manually handle the spaces.

Maybe something along the lines of

def to_query_string(p, s=''):
    for k in p:
        v = p[k]
        if isinstance(v, str):
            s += f'{k}={v}&'.replace(' ', '%20')
        elif isinstance(v, int):
            s += f'{k}={v}&'
        elif isinstance(v, list):
            for i in v:
                s += f'{k}={i}&'
    return s[:-1]  # remove last '&'

which can be used as

min = 10
max = 30
params = {'query': f'score between {min} and {max}', 'limit': 1, 'information': ['name', 'location']}
response = get('/api/dogs', params=to_query_string(params))

Answer 2

We can use urllib2.Request to call url

import urllib2

send_params = {'key1': 'value  1', 'key2': 'value 2'}
new_send_params = []
for (k, v) in send_params.items():
    new_send_params.append(k + "=" + urllib2.quote(v))

url = 'http://example.com/service?'+ '&'.join(new_send_params)
req = urllib2.Request(url)
response = urllib2.urlopen(req)
print "Request URL: " + url
#Request URL: http://example.com/service?key1=value%20&key2=value%202
print response.read()
#Python Master Request handler 2016-07-04 16:05:19.928132 . Your request path is  /service?key1=value%20&key2=value%202

Answer 3

PYTHON 2.7

• Override the urllib.quote_pluse with urllib.quote

• The urlencoder uses urllib.quote_pluse to encode the data.

---

code

import requests
import urllib
urllib.quote_plus=urllib.quote # A fix for urlencoder to give %20 
payload = {'key1': 'value  1', 'key2': 'value 2'}
headers = {'Content-Type': 'application/json;charset=UTF-8'}
param = urllib.urlencode(payload) #encodes the data
r = requests.get("http://example.com/service", params=param, headers=headers, 
             auth=("admin", "password"))

output

the output for param = urllib.urlencode(payload)
'key2=value%202&key1=value%20%201' 

Answer 4

To follow up on @WeaselFox's answer, they introduced a patch that accepts a quote_via keyword argument to urllib.parse.urlencode. Now you could do this:

import requests
import urllib

payload = {'key1': 'value  1', 'key2': 'value 2'}
headers = {'Content-Type': 'application/json;charset=UTF-8'}
params = urllib.parse.urlencode(payload, quote_via=urllib.parse.quote)
r = requests.get("http://example.com/service", params=params, headers=headers,
    auth=("admin", "password"))

Answer 5

this seems to be a known bug/issue in python :

http://bugs.python.org/issue13866

I think you will have to go around this issue using urllib and urllib2 and avoid requests. look at the bug reports for some tips on how to do that.

Answer 6

I only find urllib.parse.quote , which can replace space to %20 .

But quote could not convert a dict.

so, We must use quote to transform dict in advance.

---

#for python3
from urllib.parse import quote

payload = {'key1': 'value  1', 'key2': 'value 2'}

newpayload = {}
for (k, v) in payload.items():
    newpayload[quote(k)] = quote(v)
print(newpayload)
#print result: {'key1': 'value%20%201', 'key2': 'value%202'}
# Now, you can use it in requests