How to protect/monitor your site from crawling by malicious user

Question

Situation:

• Site with content protected by username/password (not all controlled since they can be trial/test users)
• a normal search engine can\'t get at i

Answer 1

The problem with option 3 is that the auto-logout would be trivial to avoid once the scraper figures out what is going on.