问题
I found an XHR request for a website, now I want to convert this request to corresponding python/requests code and struggled at how to parse the responseType
to corresponding replacement code, here is the download script:
const download = (url, params = {}, dowType, name, img, method) =>
new Promise((resolve, reject) => {
const _tmpArr = Object.keys(params) || [];
const _formData = new FormData();
_tmpArr.map(value => {
if (params[value] || params[value] === 0 || params[value] === '') {
_formData.append(value, params[value]);
}
});
let xhr = new XMLHttpRequest();
xhr.open('POST', url, true);
xhr.responseType = 'blob';
xhr.onload = function(e) {
if (this.status === 200) {
const reader = new FileReader();
reader.onload = event => {
//....
};
reader.readAsText(this.response);
}
if (!img) {
resolve();
}
};
xhr.send(_formData);
});
download('http://the/post/url', {
sampleNo
})
Since I didn't write javascript code mainly and didn't get any helps from mozilla docs about the blob, then I copied to the corresponding curl
command via Copy as cURL
function in Chrome's network inspector, here is the command text:
curl 'http://the/post/url' \
-H 'Connection: keep-alive' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36' \
-H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundarybfGJ0I90Seu9Ywjc' \
-H 'Accept: */*' \
-H 'Origin: http://www.example.com' \
-H 'Referer: http://www.example.com/reports/' \
-H 'Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7,zh-TW;q=0.6' \
--data-binary $'------WebKitFormBoundarybfGJ0I90Seu9Ywjc\r\nContent-Disposition: form-data; name="sampleNo"\r\n\r\nXFFFFFF000000\r\n------WebKitFormBoundarybfGJ0I90Seu9Ywjc--\r\n' \
--compressed \
--insecure
For the following version of python code, it sends the post request with multipart-formdata to the service and a logic service error returns (the status code is 200, but the response data is {'code': '000001', 'msg': 'internal system exception', 'data': 'getInputStream() has already been called for this request'}
, this is a Java related exception from server side and it confirms the request data shall(?) get received).
I tries to update the Accept
header value to blob
, */blob
, application/octet-stream
, application/python-pickle
and others, none of them work, it returns 500 error directly. So I wonder what's the right replacement about the responseType
with blob
in XHR request.
import os
import logging
import requests
from requests_toolbelt import MultipartEncoder
logging.basicConfig(level=logging.DEBUG)
def download(order_id):
url = "http://the/post/url"
m = MultipartEncoder(fields={'sampleNo': order_id})
headers = {
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36',
'Accept': '*/*',
'Origin': 'http://www.example.com',
'Referer': 'http://www.example.com/reports/.....',
'Accept-Language': 'en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7,zh-TW;q=0.6',
'dnt': '1',
'Content-Type': m.content_type,
}
try:
logging.info('Downloading report %s' % order_id)
response = requests.post(url, data=m, headers=headers)
if response.status_code != requests.codes.OK:
logging.error(response.content)
return False
json_data = response.json()
if json_data.get('code') != '000000':
logging.error(json_data)
return False
with open(filepath, 'wb') as f:
f.write(response.content)
logging.info('Downloaded report to %s' % filepath)
except Exception as e:
logging.exception(e)
return False
return True
来源:https://stackoverflow.com/questions/62538257/convert-xmlhttprequest-to-python-requests