copy file from Google Drive to Google Cloud Storage within Google

Question

Is it possible to copy a file from Google Drive to Google Cloud Storage? I imagine it would be very fast since both are on a similar storage system.

I haven\'t seen

Answer 1

@frunkad commented a link to a nice workaround, but for completeness I will recite this here, as it is currently the top result in search.

You can open a Colab (Juypiter Notebook on Google servers), mount your gDrive and use the gCloud CLI to copy files.

1. open https://colab.research.google.com/ and connect to a machine
2. Mount your Drive by executing the code below, clicking on the link and pasting the authentification key

from google.colab import drive
drive.mount(‘/content/drive’)

3. Connect to your Google Cloude Storage. (Project id can be found here

from google.colab import auth
auth.authenticate_user()project_id = 'your-project-id'
!gcloud config set project {project_id}
!gsutil ls

4. Copy files using gsutil. Use -m tag for multi-threading to increase speed. (There is a Subfolder called "My Drive" that you have to address in your mounted drive)

bucket_name = 'your_bucket_name'
!gsutil -m cp -r /content/drive/My\ Drive/Your-Data/* gs://{bucket_name}/

Original Author Philip Lies

Link to his colab: https://colab.research.google.com/drive/1Xc8E8mKC4MBvQ6Sw6akd_X5Z1cmHSNca