HTTP redirection loop RuntimeError in open_uri_redirections gem

血红的双手。 提交于 2019-12-12 22:19:57

问题


Thanks for your time.

I'm working up a Ruby script to parse a CSV of urls and evaluate them on a variety of dimensions to see if certain tags and attributes are present or confirm to certain pattern.

I'm using Nokogiri, open-uri and the patch for open-url to allow the script to follow redirections, open_uri_redirections.

On a handful of problematic domains, I encounter an error and the script encounters a runtime error:

Loading https://www.exampleproblemdomain.com C:/Ruby24-x64/lib/ruby/2.4.0/open-uri.rb:233:in `open_loop': HTTP redirection loop: https://www.exampleproblemdomain.com (RuntimeError)
        from C:/Ruby24-x64/lib/ruby/2.4.0/open-uri.rb:151:in `open_uri'
        from C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/open_uri_redirections-0.2.1/lib/open-uri/redirections_patch.rb:55:in `open_uri'
        from C:/Ruby24-x64/lib/ruby/2.4.0/open-uri.rb:721:in `open'
        from C:/Ruby24-x64/lib/ruby/2.4.0/open-uri.rb:35:in `open'
        from arraycsv.rb:16:in `block in <main>'
        from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:866:in `each'
        from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:866:in `each'
        from arraycsv.rb:14:in `<main>'

The open_uri_redirections gem works well for most domains but a select few encounter this issue and I can't figure out why, even when I use a redirection extension like Ayima to follow the chain of redirections.

Could it be a 302 vs. a 301 creating the issue?

来源:https://stackoverflow.com/questions/54411704/http-redirection-loop-runtimeerror-in-open-uri-redirections-gem

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!