how to identify web crawlers of google/yahoo/msn by PHP?

Question

AFAIK,

$_SERVER[\'REMOTE_HOST\'] should end up with \"google.com\" or \"yahoo.com\".

but is it the most ensuring method?

any other way out?

Answer 1

Google/Bing/Yahoo Crawlers IP Addresses -

http://myip.ms/info/bots/Google_Bing_Yahoo_Facebook_etc_Bot_IP_Addresses.html

Answer 2

I hacked something together, but you will have to look at $_SERVER['HTTP_USER_AGENT'] to see if they come from a search-engine domain.

function is_crawlers() {

   $sites = 'Google|Yahoo|msnbot|'; // Add the rest of the search-engines 

   return (preg_match("/$sites/", $_SERVER['HTTP_USER_AGENT']) > 0) ? true : false;  

   }