A few techniques, including the ones you've mentioned:
- Level of detail. Outlined above.
- Mipmapping. For textures.
- Field of view. Restrict how many things your camera can 'see' by picking an appropriate zoom level. See also view frustrum culling.
- Far-clipping plane. Occlusion of far-off places: most games use fog, mountains, or some other excuse to have a hard cut-off on how far away you can see. This can be implemented using octrees, or some other such technique. This may be based on whole objects, or polygons, or whatever, and is usually set to occur right behind those aforementioned mountains.
- Arenas/levels/areas/whatever -- again most games artificially restrict you to playing in one arena at a time; the benefit is obvious (you get to ignore objects in all other arenas)
- Pre-rendering -- this is what you're suggesting above. For static scenes that are far off, render the scene in a 'typical' view and make it a texture on the background, or use it for the skybox / reflection map / etc.
The idea is to throw away as much detail as is possible before users start to notice.
There are a number of pre-existing toolkits that will do these things automatically, but they tend to cost money. If you're looking at a serious app, I would recommend at least researching the solutions they include.