How to import modules in Google App Engine?

Question

I have created a simple GAE app based on the default template. I want to add an external module like short_url. How do I do this? The directions that I have found so far are

Answer 1

You can import python packages as ZIPs. This allows you to avoid the maximum file count.

The app engine docs address this.

python 2.5: zipimport is supported.

python 2.7: zipimport is not supported, but Python 2.7 can natively import from .zip files.

This is how I import boto.

sys.path.insert(0, 'boto.zip')
import boto #pylint: disable=F0401
from boto import connect_fps  #pylint: disable=F0401

The cons of this technique include having to manually re-archive many packages.

For example, boto.zip decompresses into the "boto" subdirectory, with the "boto" module inside of it (as another subdirectory).

So to import boto naturally you may have to do from boto import boto, but this can cause weirdness with a lack of __init__.py.

To solve this, simply decompress, and archive the boto subfolder manually as boto.zip, and place that in your application folder.

Answer 2

i will second the answers given by @Adam Bernier and @S.Mark, although adam's explains things is a bit more detail. in general, you can add any pure Python module/package to your App Engine directory and use as-is, as long as they don't try to work outside of the sandbox, i.e, cannot create files, cannot open network sockets, etc.

also keep in mind the hard limits:

• maximum total number of files (app files and static files): 3,000
• maximum size of an application file: 10 megabytes
• maximum size of a static file: 10 megabytes
• maximum total size of all application and static files: 150 megabytes

UPDATE (Oct 2011): most of these numbers have been increased to:

• maximum total number of files (app files and static files): 10,000
• maximum size of an application file: 32MB
• maximum size of a static file: 32MB

UPDATE (Jun 2012): the last limit was bumped up to:

• maximum total size of all application and static files: 1GB

Answer 3

Simply place the short_url.py file in your app's directory.

Sample App Engine project:

myapp/
    app.yaml
    index.yaml
    main.py
    short_url.py
    views.py

And in views.py (or wherever), you can then import like so:

import short_url

---

For more complex projects, perhaps a better method is to create a directory especially for dependencies; say lib:

myapp/
    lib/
        __init__.py
        short_url.py
    app.yaml
    index.yaml
    main.py
    views.py

from lib import short_url

---

Edit #2:
Apologies, I should have mentioned this earlier. You need modify your path, thanks to Nick Johnson for the following fix.
Ensure that this code is run before starting up your app; something like this:

import os
import sys

def fix_path():
    # credit:  Nick Johnson of Google
    sys.path.append(os.path.join(os.path.dirname(__file__), 'lib'))

def main():
    url_map = [ ('/', views.IndexHandler),] # etc.
    app = webapp.WSGIApplication(url_map, debug=False)
    wsgiref.handlers.CGIHandler().run(app)

if __name__ == "__main__":
    fix_path()
    main()

---

Edit3:
To get this code to run before all other imports, you can put the path managing code in a file of its own in your app's base directory (Python recognizes everything in that directory without any path modifications).
And then you'd just ensure that this import

import fix_path

...is listed before all other imports in your main.py file.
Here's a link to full, working example in case my explanation wasn't clear.

Answer 4

Since that url_shortener program written in python, you could just include in your source code and import it like other python modules.