Is there any risk if I allow only one User Agent in robots.txt (‘Screaming Frog SEO Spider’) to crawl my website?

前端 未结 0 802
一整个雨季
一整个雨季 2021-01-19 19:03

We didn\'t launch our site yet, it is on a testing server under password with all user agents blocked in robots.txt. We want to run an audit before launching and in order to

相关标签:
回答
  • 消灭零回复
提交回复
热议问题