Check if two urls are for the same website

依然范特西╮ 提交于 2021-02-11 12:51:45

问题


I'm looking for a way to compare two urls. I can do:

URI('http://www.test.com/blabla').host

to have the base name, but this not reliable. For example:

URI('http://www.test.com/blabla').host == URI('http://test.com/blabla').host

returns false, but they can be the same site. To have the IP address is not reliable too because if I do:

IPSocket.getaddress(URI('http://hello.herokuapp.com').host) ==
IPSocket.getaddress(URI('http://test.herokuapp.com').host)

It returns true, but they are not the same site. Is there a more reliable way?


回答1:


The site under http://foo.com can be the same as under http://www.foo.com, but it can be a totally different site, due to web server configuration. It depends on the DNS config too, which IP points to www and which one to without www.

If you want compare two sites, you need to fetch the content, and compare key parts (using nokogiri for example) about similarities.

Nowadays due to sidebars and news, two consequent request to the same url, gives slight different html responses.



来源:https://stackoverflow.com/questions/22283240/check-if-two-urls-are-for-the-same-website

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!