How to import data from Google Cloud Storage to Google Colab

Question

Currently I am working on a data set that is of 10 GB. I have uploaded it on google cloud storage but I don\'t know how to import it in google colab.

Answer 1

from google.colab import auth
auth.authenticate_user()

Once you run this, a link will be generated, you can click on it and get the signing in done.

!echo "deb http://packages.cloud.google.com/apt gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
!apt -qq update
!apt -qq install gcsfuse

Use this to install gcsfuse on colab. Cloud Storage FUSE is an open source FUSE adapter that allows you to mount Cloud Storage buckets as file systems on Colab, Linux or macOS systems.

!mkdir folderOnColab
!gcsfuse folderOnBucket/content/ folderOnColab

Use this to mount the directories. (folderOnBucket is the GCS bucket URL without the gs:// part)

You can use this docs for further reading. https://cloud.google.com/storage/docs/gcs-fuse