How to configure robots.txt to allow everything?

前端 未结 4 656
北海茫月
北海茫月 2021-01-30 04:46

My robots.txt in Google Webmaster Tools shows the following values:

User-agent: *
Allow: /

What does it mean? I don\'t have enough

4条回答
  •  离开以前
    2021-01-30 05:11

    That file will allow all crawlers access

    User-agent: *
    Allow: /
    

    This basically allows all user agents (the *) to all parts of the site (the /).

提交回复
热议问题