information-extraction

I'm trying to figure out how to extract dates from unstructured text using Ruby. For example, I'd like to parse the date out of this string "Applications started after 12:00 A.M. Midnight (EST) February 1, 2010 will not be considered." Any suggestions? Assuming you just want dates and not datetimes: require 'date' string = "Applications started after 12:00 A.M. Midnight (EST) February 1, 2010 will not be considered." r = /(January|February|March|April|May|June|July|August|September|October|November|December) (\d+{1,2}), (\d{4})/ if string[r] date =Date.parse(string[r]) puts date end Try

I have question about how to evaluate the information retrieve result is good or not such as calculate the relevant document rank, recall, precision ,AP, MAP..... currently, the system is able to retrieve the document from the database once the users enter the query. The problem is I do not know how to do the evaluation. I got some public data set such as "Cranfield collection" dataset link it contains 1.document 2.query 3.relevance assesments DOCS QRYS SIZE* Cranfield 1,400 225 1.6 May I know how to use do the evaluation by using "Cranfield collection" to calculate the relevant document rank,

I have a document with tagged data in the format Hi here's my [KEYWORD phone number], let me know when you wanna hangout: [PHONE 7802708523]. I live in a [PROP_TYPE condo] in [CITY New York] . I want to train a model based on a set of these type of tagged documents, and then use my model to tag new documents. Is this possible in NLTK? I have looked at chunking and NLTK-Trainer scripts, but these have a restricted set of tags and corpora, while my dataset has custom tags. As @AleksandarSavkov wrote already, this is essentially a named entity recognition (NER) task-- or more generally a chunking

I am new to text mining. I am using a open source jar (Mate Parser) which gives me output in a CoNLL 2009 format after dependency parsing. I want to use the dependency parsing results for Information Extraction. But i am able to understand some of the output but not able to comprehend the CoNLL data format. Can any one help me in making me understand the CoNLL data format?? Any kind of pointers would be appreciated. There are many different CoNLL formats since CoNLL is a different shared task each year. The format for CoNLL 2009 is described here . Each line represents a single word with a

I'm looking for a PDF library which will allow me to extract the text from a PDF document. I've looked at PyPDF, and this can extract the text from a PDF document very nicely. The problem with this is that if there are tables in the document, the text in the tables is extracted in-line with the rest of the document text. This can be problematic because it produces sections of text that aren't useful and look garbled (for instance, lots of numbers mashed together). I'm looking for something that's a bit more advanced. I'd like to extract the text from a PDF document, excluding any tables and

I am extracting data from a forum. My script based on is working fine. Now I need to extract date and time (21 Dec 2009, 20:39) from single post. I cannot get it work. I used FireXPath to determine the xpath. Sample code: require 'rubygems' require 'mechanize' post_agent = WWW::Mechanize.new post_page = post_agent.get('http://www.vbulletin.org/forum/showthread.php?t=230708') puts post_page.parser.xpath('/html/body/div/div/div/div/div/table/tbody/tr/td/div[2]/text()').to_s.strip puts post_page.parser.at_xpath('/html/body/div/div/div/div/div/table/tbody/tr/td/div[2]/text()').to_s.strip puts post

In the iOS email client, when an email contains a date, time or location, the text becomes a hyperlink and it is possible to create an appointment or look at a map simply by tapping the link. It not only works for emails in English, but in other languages also. I love this feature and would like to understand how they do it. The naive way to do this would be to have many regular expressions and run them all. However I this is not going to scale very well and will work for only a specific language or date format, etc. I think that Apple must be using some concept of machine learning to extract