Will this robots.txt only allow googlebot to index my site?

后端 未结 3 1677

Will this robots.txt file only allow googlebot to index my site\'s index.php file? CAVEAT, I have an htaccess redirect that people who type in

http://www.example

3条回答
  •  慢半拍i
    慢半拍i (楼主)
    2021-01-27 13:32

    Absent the redirect, Googlebot would not see your site, except for the index.php.

    With the redirect, it depends on how the bot handles redirects and how your htaccess does the redirect. If you return a 302, then Googlebot will see http://www.example.com/, check against robots.txt, and not see the main site. Even if you do an internal redirect and tell Googlebot that the responding page is http://www.example.com/, it will see the page but might not index it.

提交回复
热议问题