Can't find kaggle.json file in google colab

问题

I'm trying to download the kaggle imagenet object localization challenge data into google colab so that I can use it to train my model. Kaggle uses an API for easy and fast access to their datasets. (https://github.com/Kaggle/kaggle-api) However, when calling the command "kaggle competitions download -c imagenet-object-localization-challenge" in google colab, it can't find the kaggle.json file which contains my username and api-key.

I haven't had this problem on my mac when running a jupyter notebook, but since I want to use google's gpu for my model, I started using google colab. Because the kaggle API expects the username and api-key to be in a kaggle.json file located in a .kaggle directory, I first created the directory .kaggle and then the file kaggle.json, into which I wrote my username and api-key (The example below doesn't display my username and api-key). I then tried to configure the path to my json file for kaggle to use when calling the kaggle download command.

!pip install kaggle

!mkdir .kaggle
!touch .kaggle/kaggle.json

api_token = {"username":"username","key":"api-key"}

import json
import zipfile
import os
with open('/content/.kaggle/kaggle.json', 'w') as file:
    json.dump(api_token, file)

!chmod 600 /content/.kaggle/kaggle.json
!kaggle config path -p /content

However, when running the last command, I got the error:

IOError: Could not find kaggle.json. Make sure it's located in /root/.kaggle. Or use the environment method.

My goal was to use the following commands to get the dataset from kaggle:

!kaggle competitions download -c imagenet-object-localization-challenge
os.chdir('/content/competitions/imagenet-object-localization-challenge')
for file in os.listdir():
    zip_ref = zipfile.ZipFile(file, 'r')
    zip_ref.extractall()
    zip_ref.close()

I don't understand why the kaggle API can't find my json file. How can I use the API in google colab?

回答1:

As the error said, you need to put kaggle.json in the right place.

Try:

!mv .kaggle /root/

Then run your code again.

回答2:

According to kaggle api documentation the location where credentials json is looking for is ~/.kaggle/kaggle.json as google colab environment is Linux based. In your snippet you try to config path parameter, but it is not used to looking for credential json:

- path: Folder where file(s) will be downloaded, defaults to current working directory

So the full working snippet for google colab environment would be:

!mkdir ~/.kaggle
!touch ~/.kaggle/kaggle.json

api_token = {"username":"username","key":"api-key"}

import json

with open('/root/.kaggle/kaggle.json', 'w') as file:
    json.dump(api_token, file)

!chmod 600 ~/.kaggle/kaggle.json

And then some api call like

!kaggle datasets download -d datamunge/sign-language-mnist

回答3:

In case you are a windows user, move your kaggle.json into .kaggle folder, that you are able to find it in your system root (which is in your username folder).

回答4:

You can find the JSON file that you need to place form Kaggle's portal itself. Just click 'Create New API Token' and it will give you a JSON file that needs to be placed.

来源：https://stackoverflow.com/questions/56816537/cant-find-kaggle-json-file-in-google-colab

标签

python

google-colaboratory

kaggle