Block some of dynamic pages from search engines

问题

I need to block some of my pages from search engines.How can I do that ? App has been developed by using ASP.net MVC and AngularJS. Thanks in advance.

These are the urls which I want to block from the search engines.

http://localhost:12534/myurl123-event?participant=12957

http://localhost:12534/myurl123-event

Note : The last part of the url is dynamic (i.e. myurl123-event?participant=12957 and myurl123-event).

回答1:

You can use a robots.txt with a disallow setting:

User-agent: *
Disallow: /myurl123-event

If you want to block everything, then you can use

Disallow: /*

or you can put all your dynamic pages inside a route (or use the controller name, if you're routing is straightforward controller/method/id)

Disallow: /dynamic/

回答2:

The Google preferred way is to use canonical links:

<link rel="canonical" href="http://somedomain.com/myurl123-event" />

This tag's function is to tell the search engine what URL should be indexed on these pages containing dynamic querystring params.

See more about them here: Google link

Using these you wont have to worry about accidentally blacklisting part of your site, such as with using robots.txt files and taking a hit on your traffic.

来源：https://stackoverflow.com/questions/31943061/block-some-of-dynamic-pages-from-search-engines