Create CKAN dataset using CKAN API and Python Requests library

Question

I am using CKAN version 2.2 and am trying to automate dataset creation and resource upload. I seem to be unable to create a dataset using the python requests libra

Answer 1

The data you send must be JSON encoded.

From the documentation (the page you linked to):

To call the CKAN API, post a JSON dictionary in an HTTP POST request to one of CKAN’s API URLs.

In the urllib example this is performed by the following line of code:

data_string = urllib.quote(json.dumps(dataset_dict))

I think (though you should check) that the requests library will do the quoting for you - so you just need to convert your dict to JSON. Something like this should work:

r = requests.post(d_url, data=json.dumps(dataset_dict), headers=auth)

Answer 2

I finally came back to this and figured it out. Alice's suggestion to check the encoding was very close. While requests does do the encoding for you, it also decides on its own which type of encoding is appropriate depending on the inputs. If a file is passed in along with a JSON dictionary, requests automatically does multipart/form-data encoding which is accepted by CKAN therefore the request is successful.

However if we pass only a JSON dictionary the default encoding is form encoding. CKAN needs requests without files to be URL encoded (application/x-www-form-urlencoded). To prevent requests from doing any encoding we can pass our parameters in as a string then requests will perform only a POST. This means we have to URL encode it ourselves.

Therefore if I specify the content type, convert the parameters to a string and encode with urllib and then pass the parameter to requests:

head['Content-Type'] = 'application/x-www-form-urlencoded'
in_dict = urllib.quote(json.dumps(in_dict))
r = requests.post(url, data=in_dict, headers=head)

Then the request is successful.