You can use Seo technique...
There is code you can put into your HTML which will prevent the bots from spidering your page(s). This code is put in the header of your web page(s) (meaning between the and tags), in the form of a meta tag (and you thought meta tags were for keywords only).
Here are four examples of how this code will look;
<meta name="robots" content="index,follow">
<meta name="robots" content="noindex,follow">
<meta name="robots" content="index,nofollow">
<meta name="robots" content="noindex,nofollow">
Add a file robots.txt
in your website root directory with the following contents:
User-agent: *
Disallow: /users/download/id
This approach basically tells the Search Engine not to index /users/download/id
.
This doesn't mean every Search Engine will do that, but most of the modern Search Engines will take in consideration that file.
<meta name="googlebot" content="noindex,nofollow,noarchive,nosnippet,noodp" />
<meta name="robots" content="noindex,nofollow,noarchive,nosnippet,noodp" />