pickling python objects to google cloud storage

Question

I\'ve been pickling the objects to filesystem and reading them back when needed to work with those objects. Currently I\'ve this code for that purpose.

def p         


        

Answer 1

For Python 3 users, you can use gcsfs library from Dask creator to solve your issue.

Example reading :

import gcsfs

fs = gcsfs.GCSFileSystem(project='my-google-project')
fs.ls('my-bucket')
>>> ['my-file.txt']
with fs.open('my-bucket/my-file.txt', 'rb') as f:
    print(f.read())

It basically is identical with pickle tho :

with fs.open(directory + '/' + filename, 'wb') as handle:
        pickle.dump(shandle)

To read, this is similar, but replace wb by rb and dump with load :

with fs.open(directory + '/' + filename, 'rb') as handle:
        pickle.load(handle)