Ruby+Anemone Web Crawler: regex to match URLs ending in a series of digits

纵然是瞬间 提交于 2019-12-04 13:28:28
  Anemone.crawl(url, :depth_limit => 3, :obey_robots_txt => true, :skip_query_strings => true) do |anemone|
   anemone.on_every_page do |page|
     pURL = page.url.to_s
     puts "Now checking: " + pURL
      bestGuess[pURL] = match_freq( manList, page.doc.inner_text )
     puts "Successfully checked"
   end
 end

Actually the /\?.*\d+$/ works:

~> irb
> all systems are go wirble/hirb/ap/show <
ruby-1.9.2-p180 :001 > "http://hiddenwebsite.com/anonimize/index.php?page=press_and_news&subpage=20060117".match /\?.*\d+$/
 => #<MatchData "?page=press_and_news&subpage=20060117"> 
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!