Multiple Sitemap: entries in robots.txt?

Question

I have been searching around using Google but I can\'t find an answer to this question.

A robots.txt file can contain the following line:

Sitemap: http:/         


        

Answer 1

You need specify in your in your file sitemap.xml this code

<?xml version="1.0" encoding="UTF-8"?>
   <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.exemple.com/sitemap1.xml.gz</loc>
   </sitemap>
   <sitemap>
      <loc>http://www.exemple.com/sitemap2.xml.gz</loc>
   </sitemap>
   </sitemapindex>

source https://support.google.com/webmasters/answer/75712?hl=fr#

Answer 2

It is possible to write them, but it is up to the search engine to know what to do with it. I suspect many search engines will either "keep digesting" more and more tokens, or alternatively, take the last sitemap they find as the real one.

I propose that the question be "if I want ____ search engine to index my site, would I be able to define multiple sitemaps?"

Answer 3

If your sitemap is over 10 MB (uncompressed) or has more than 50 000 entries Google requires that you use multiple sitemaps bundled with a Sitemap Index File.

• Using Sitemap index files (to group multiple sitemap files)

In your robots.txt point to a sitemap index which should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.example.com/sitemap1.xml.gz</loc>
      <lastmod>2012-10-01T18:23:17+00:00</lastmod>
   </sitemap>
   <sitemap>
      <loc>http://www.example.com/sitemap2.xml.gz</loc>
      <lastmod>2012-01-01</lastmod>
   </sitemap>
</sitemapindex>

Answer 4

It's recommended to create a sitemap index file, rather separate XML URLs to put in your your robots.txt file.

Then, put the indexed sitemap URL as below in your robots.txt file.

Sitemap: http://www.yoursite.com/sitemap_index.xml

If you want to learn how to create indexed sitemap URL, then follow this guide from sitemap.org

Best Practice:

• Create image sitemap, video sitemap separately if your website has huge number of such contents.
• Check spelling of robots file, it should be robots.txt, don't use robot.txt or any misspelling. Put robots.txt file in root directly only.
• For more info, you can visit robots.txt's official website.

Answer 5

Yes it is possible to have more than one sitemap-index-file:

You can have more than one Sitemap index file.

Highlight by me.

Yes it is possible to list multiple sitemap-files within robots.txt, see as well in the sitemap.org site:

You can specify more than one Sitemap file per robots.txt file.

Sitemap: http://www.example.com/sitemap-host1.xml

Sitemap: http://www.example.com/sitemap-host2.xml

Highlight by me, this can not be misread I'd say, so simply spoken, this can be done.

This is also necessary for cross-submits, for which btw. the robots.txt has been chosen.

Btw Google, Yahoo and Bing, all are members of sitemaps.org:

Sitemap 0.90 is offered under the terms of the Attribution-ShareAlike Creative Commons License and has wide adoption, including support from Google, Yahoo!, and Microsoft.

So you can rest assured that your sitemap entries will be properly read by the search engine bots.

Submitting them via webmaster tools can not hurt either - as John Mueller commented.