How may i prevent search engines from crawling a subdomain on my website?

问题

I have cPanel installed on my website.

1. I went to the Domains section on cPanel

2. I clicked on subdomains.

3. I assigned the subdomain name (e.g : personal.mywebsite.com )

4. It wanted me to assign document root folder also. I assigned mywebsite.com/personal

   if i create robots.txt in my website root(e.g : website.com)

   User-agent:

   Disallow: /personal/

Can it also block personal.mywebsite.com?

what should i do?

thanks

回答1:

When you want to block URLs on personal.example.com, visit http://personal.example.com/robots.txt (resp. https instead of http).

It doesn’t matter how your server organizes folders in the backend, it only matters which robots.txt is available when accessing this URL.

来源：https://stackoverflow.com/questions/24571665/how-may-i-prevent-search-engines-from-crawling-a-subdomain-on-my-website