Stop abusive bots from crawling?

Question

Is this a good idea??

http://browsers.garykeith.com/stream.asp?RobotsTXT

What does abusive crawling mean? How is that bad for my

Answer 1

A user-agent (which includes crawlers) is under no obligation to honour your robots.txt. The best you can do is try to identify abusive access patterns (via web-logs, etc.), and block the corresponding IP.