How to prevent Googlebot from overwhelming site?

Question

I\'m running a site with a lot of content, but little traffic, on a middle-of-the-road dedicated server.

Occasionally, Googlebot will stampede us, resulting in Apache

Answer 1

Note that you can set the crawl speed via Google Webmaster Tools (under Site Settings), but they only honour the setting for six months! So you have to log in every six months to set it again.

This setting was changed in Google. The setting is only saved for 90 days now (3 months, not 6).