Google Cloud Storage on Appengine Dev Server

Question

There\'s a similar question that was recently responded to on Stackoverflow here: Google Cloud Storage Client not working on dev appserver

The solution was to either

Answer 1

The storage simulator for the local server is working in later versions of the SDK. For Java, one may choose to follow a dedicated tutorial: “App Engine and Google Cloud Storage Sample”.

Answer 2

To access gcs objects on dev_appserver, you must specify the bucket & object name, i.e. /_ah/gcs/[bucket]/[object].

Answer 3

I was able to find the google cloud storage files I wrote to a bucket locally at:

    localhost:port/_ah/gcs/bucket_name/file_suffix

Where port is by default 8080, and the file was written to: /bucket_name/file_suffix

For those trying to understand the full process of setting up a simple python GAE app and testing local writes to google cloud storage:

1. Follow the google app engine "quickstart":

https://cloud.google.com/appengine/docs/standard/python/quickstart

2. Run a local dev server with:

    dev_appserver.py app.yaml 

3. If using python, follow "App Engine and Google Cloud Storage Sample":

https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/app-engine-cloud-storage-sample

If you run into "ImportError: No module named cloudstorage" you need to create a file named appengine_config.py

    touch appengine_config.py

and add to it:

    from google.appengine.ext import vendor
    vendor.add('lib')

GAE runs this script automatically when starting your local dev server with dev_appserver.py app.yaml, and it is necessary to run this script for GAE to find the cloudstorage library in your lib/ folder

4. "Writing a file to cloud storage" from the same tutorial:

    def create_file(self, filename):
    """Create a file."""

       self.response.write('Creating file {}\n'.format(filename))

       # The retry_params specified in the open call will override the default
       # retry params for this particular file handle.
       write_retry_params = cloudstorage.RetryParams(backoff_factor=1.1)
       with cloudstorage.open(
           filename, 'w', content_type='text/plain', options={
               'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
               retry_params=write_retry_params) as cloudstorage_file:
                   cloudstorage_file.write('abcde\n')
                   cloudstorage_file.write('f'*1024*4 + '\n')
       self.tmp_filenames_to_clean_up.append(filename) 

       with cloudstorage.open(
           filename, 'w', content_type='text/plain', options={
               'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
               retry_params=write_retry_params) as cloudstorage_file:
                   cloudstorage_file.write('abcde\n')
                   cloudstorage_file.write('f'*1024*4 + '\n')

Where filename is /bucket_name/file_suffix

4. After calling create_file via a route in your WSGI app, your file will be available at:

    localhost:port/_ah/gcs/bucket_name/file_suffix

Where port is by default 8080, and the file was written to: /bucket_name/file_suffix

Postscript

Unfortunately, I did not find either 3) or 4) in their docs, so I hope this helps someone get set up more easily in the future.