I've found a great API for reading Openstreetmap data; Mapscript.
This is an interface to MapServer, which is able to read *.map and *.shp files. It is able to use a spatial index and is therefore potentially extremely fast. I've done random polygon lookups (checking if a polygon intersects land) of the world coastline shape file in 40 ms.