lemmatization

The following code prints out leaf : from nltk.stem.wordnet import WordNetLemmatizer lem = WordNetLemmatizer() print(lem.lemmatize('leaves')) This may or may not be accurate depending on the surrounding context, e.g. Mary leaves the room vs. Dew drops fall from the leaves . How can I tell NLTK to lemmatize words taking surrounding context into account? TL;DR First tag the sentence, then use the POS tag as the additional parameter input for the lemmatization. from nltk import pos_tag from nltk.stem import WordNetLemmatizer wnl = WordNetLemmatizer() def penn2morphy(penntag): """ Converts Penn

Given some (English) word that we shall assume is a plural , is it possible to derive the singular form? I'd like to avoid lookup/dictionary tables if possible. Some examples: Examples -> Example a simple 's' suffix Glitch -> Glitches 'es' suffix, as opposed to above Countries -> Country 'ies' suffix. Sheep -> Sheep no change: possible fallback for indeterminate values Or, this seems to be a fairly exhaustive list. Suggestions of libraries in language x are fine, as long as they are open-source (ie, so that someone can examine them to determine how to do it in language y ) It really depends on

When do I use each ? Also...is the NLTK lemmatization dependent upon Parts of Speech? Wouldn't it be more accurate if it was? Short and dense: http://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html The goal of both stemming and lemmatization is to reduce inflectional forms and sometimes derivationally related forms of a word to a common base form. However, the two words differ in their flavor. Stemming usually refers to a crude heuristic process that chops off the ends of words in the hope of achieving this goal correctly most of the time, and often includes the

I'm working on a lemmatizer using python, NLTK and the WordNetLemmatizer. Here is a random text that output what I was expecting from nltk.stem import WordNetLemmatizer from nltk.corpus import wordnet lem = WordNetLemmatizer() lem.lemmatize('worse', pos=wordnet.ADJ) // here, we are specifying that 'worse' is an adjective Output: 'bad' lem.lemmatize('worse', pos=wordnet.ADV) // here, we are specifying that 'worse' is an adverb Output: 'worse' Well, everything here is fine. The behaviour is the same with other adjectives like 'better' (for an irregular form) or 'older' (note that the same test