How do App Engine version numbers work? Are they only integers? Can I use floats? Can I reuse old version numbers?
In order to access a version of your app other than the default one, add #. before your app url as explained here: https://stackoverflow.com/a/8549546/129202 This is useful for testing new versions before you make them live.
If the version of app is 2 and your app URL is appname.appspot.com then URL 2.appname.appspot.com will give you access to version 2 of your app.
You can increment it when you pushing a new release. See below for reference.
When you build a new major release of an application that is already running on App Engine, you can upload the new release as a new version. The old version will continue to serve users until you switch to the new version. You can test the new version on App Engine while the old version is still running.
Edited: Didn't remove my answer because there're some good comments on it.
From the manual:
http://code.google.com/appengine/docs/python/config/appconfig.html#Required_Elements
A version specifier for the application code. App Engine retains a copy of your application for each version used. An administrator can change which major version of the application is default using Administration Console, and can test non-default versions before making them default. The version specifier can contain letters, digits, and hyphens.
Each version of an application retains its own copy of app.yaml. When an application is uploaded, the version mentioned in the app.yaml file being uploaded is the version that gets created or replaced by the upload.