How to encode parameter as gbk instead of utf-8 with cURL or python requests?

问题

I have a strange API, it just accept gbk parameters, I capture the data in Windows IE browser, show data with this command:

$ cat 12_Request.txt| iconv -f GBK -t UTF-8

GET http://10.202.15.197:20176/?user_id=1&query_type=GEOSPLIT&address=广东省深圳市宝安&ret_splitinfo=1 HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: zh-CN
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: 10.202.15.197:20176
DNT: 1
Connection: Keep-Alive

As you can see, my data is encode in GBK. Then I send data with netcat like this:

$ cat 12_Request.txt| nc 10.202.15.197 20176 | iconv -f GBK -t utf-8 # right

HTTP/1.0 200 OK
Content-Type: application/octet-stream
Connection: close
Content-Length: 222

<?xml version='1.0' encoding='GBK'?>
<addrSplitInfo>
<status>0</status><as_info prop="1" level="1">广东省</as_info>
<as_info prop="1" level="2">深圳市</as_info>
<as_info prop="3" level="18">宝安</as_info>
</addrSplitInfo>

but if I send data with UTF-8, I get the wrong response:

$ cat 12_Request.txt | iconv -f GBK -t utf-8 | nc 10.202.15.197 20176 # wrong      

HTTP/1.0 200 OK
Content-Type: application/octet-stream
Connection: close
Content-Length: 152

<?xml version='1.0' encoding='GBK'?>
<addrSplitInfo>
<status>0</status><as_info prop="6" level="13">广东省深圳市宝安</as_info>
</addrSplitInfo>

I have tried to send like this in cURL:

curl -v \
--header 'Accept: text/html, application/xhtml+xml, */*' \
--header 'Accept-Language: zh-CN' \
-A 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko' \
--header 'Accept-Encoding: gzip, deflate' \
--header 'Host: 10.202.15.197:20176' \
--header 'DNT: 1' \
--header 'Connection: Keep-Alive' \
http://10.202.15.197:20176?user_id=1&query_type=GEOSPLIT&address=广东省深圳市宝安&ret_splitinfo=1

this doesn't work, it get response as follow:

<?xml version='1.0' encoding='GBK'?>
<addrSplitInfo>
<status>0</status><as_info prop="6" level="13">广东省深圳市宝安</as_info>
</addrSplitInfo>

Then, with data-urlencode:

curl \
--header 'Accept: text/html, application/xhtml+xml, */*' \
--header 'Accept-Language: zh-CN' \
-A 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko' \
--header 'Accept-Encoding: gzip, deflate' \
--header 'Host: 10.202.15.197:20176' \
--header 'DNT: 1' \
--header 'Connection: Keep-Alive' \
http://10.202.15.197:20176 --data-urlencode 'user_id=1&query_type=GEOSPLIT&address=广东省深圳市宝安&ret_splitinfo=1'

this doesn't work, it gets response like this:

<?xml version='1.0' encoding='GBK'?>
<searchresult><status>1</status><count>0</count>
</searchresult>

Also I tried with python,

>>> import requests
>>> url = u"http://10.202.15.197:20176?user_id=1&query_type=GEOSPLIT&address=广东省深圳市宝安&ret_splitinfo=1"
>>> r = requests.get(url.encode('utf-8').decode('gbk'))
>>> print r.text

this doesn't work, the response like this:

u'<?xml version=\'1.0\' encoding=\'GBK\'?>\n<addrSplitInfo>\n<status>0</status><as_info prop="6" level="13">\xe9\u015e\x9e\u013a\xb8\xe7\u0179\u02d8\xe9\x90\u015e\xe4\u02dd\u0161\xe7\u0161\x81\xe9\x8d)\x86\u02db\xe7\u0164\u015b\xe7\x80\u0161\u0107\u017c\x86\xe7\x95\xa8</as_info>\n</addrSplitInfo>\n'

回答1:

I think I get the answer

with cURL:

echo "http://10.202.15.197:20176\?user_id\=1\&query_type\=GEOSPLIT\&address\=广东省深圳市宝安\&ret_splitinfo\=1" | iconv -f utf-8 -t gbk | xargs curl

with python

payload = {"user_id": 1, "query_type": "GEOSPLIT", "address": u"广东省深圳市宝安".encode('gbk'), "ret_splitinfo": 1}
r = requests.get("http://10.202.15.197:20176", payload)
print r.text

来源：https://stackoverflow.com/questions/43647785/how-to-encode-parameter-as-gbk-instead-of-utf-8-with-curl-or-python-requests

标签

http

curl

python-requests

netcat

gbk