Sitemap for a site with a large number of dynamic subdomains

Question

I\'m running a site which allows users to create subdomains. I\'d like to submit these user subdomains to search engines via sitemaps. However, according to the sitemaps protoco

Answer 1

I recently struggled through this and finally got it working. See this thread for more details:

http://www.google.com/support/forum/p/Webmasters/thread?tid=53c3e4b3ab8d9503&hl=en&fid=53c3e4b3ab8d9503000497bd04ba63cf

Summary:

• Use DNS verification to verify your site and all it's subdomains in one fell swoop
• make the robots.txt on all your subdomains point to the main sitemap on your www domain
• You may need to wait several days for Google to update it's cached copies of robot.txt on all your subdomains. It will still show errors until then.

Answer 2

If you have a website that allows users to create sub-domain within your site, it is better for you to simplify the process by creating and submitting sitemaps for each subdomains by creating a single sitemap. This includes sitemap URLs for all your subdomain sites and saving this sitemap to a single location. But, to do this, all sites must be verified in webmaster tools. You can define one sitemap as:

http://example.com/sitemap.xml

Define all your sub-domain sitemaps for all your sub-domain URLs under this document tree.

You can define multiple sitemap files upto 50,000 URLs and 10 megabytes file size per sitemap. Sitemaps can be compressed using gzip to reduce the bandwidth. So, you don’t have any problem by defining the sitemap in this way.

Answer 3

Yes, the subdomain restriction is in the sitemaps.org spec, but, Google has put some exceptions in place:

1. Verify all subdomains within your Google Webmaster tools account http://www.google.com/support/webmasters/bin/answer.py?answer=75712 cross-submission of sitemaps XML via Google Webmaster tools - if submitted via the root of your domain - will not throw errors for Google

2. Within the robots.txt of a subdomain you can point to sitemaps XML on other domains. there will be no cross submission errors - for Google