问题
Since some crawlers don't like the sitemap versions made for Google, I made different sitemaps.
And there is an option to put
Sitemap: http://example.com/sitemap.xml
to robots.txt. But is it possible to put it kinda like this:
User-agent: *
Sitemap: http://example.com/sitemap.xml
User-agent: googlebot
Sitemap: http://example.com/sitemap-for-google.xml
I couldn't find any resource for this topic and robots.txt is not something I want to joke around with.
回答1:
This is not possible in robots.txt. The Sitemap: value will be used by all supporting consumers.
Sitemaps.org protocol:
This directive is independent of the user-agent line, so it doesn't matter where you place it in your file.
Mentioned alternatives: submit the sitemaps to the search engines via their website or via HTTP requests.
来源：https://stackoverflow.com/questions/24973698/how-to-assign-specific-sitemaps-for-specific-crawler-bots-in-robots-txt