You can include any pure python third-party libraries with your app that you want. You don't specify them in app.yaml, that's only for third-party libraries that are included by Google on the servers.
xlrd is not "supported" per se, but it should work.