How to find out the summarized text of a given URL in python / Django? [closed]

Answer 1

I had the same need, and lemur, although it has summarization capabilities, I found it buggy to the point of being unusable. Over the weekend I used nltk to code up a summarize module in python: https://github.com/thavelick/summarize

I took the algorithm from the Java library Classifier4J here: http://classifier4j.sourceforge.net/ but used nltk and a python wherever possible.

Here is the basic usage:

>>> import summarize

A SimpleSummarizer (currently the only summarizer) makes a summary by using sentences with the most frequent words:

>>> ss = summarize.SimpleSummarizer()
>>> input = "NLTK is a python library for working human-written text. Summarize is a package that uses NLTK to create summaries."
>>> ss.summarize(input, 1)
'NLTK is a python library for working human-written text.'

You can specify any number of sentenecs in the summary as you like.

>>> input = "NLTK is a python library for working human-written text. Summarize is a package that uses NLTK to create summaries. A Summariser is really cool. I don't think there are any other python summarisers."
>>> ss.summarize(input, 2)
"NLTK is a python library for working human-written text.  I don't think there are any other python summarisers."

Unlike the original algorithm from Classifier4J, this summarizer works correctly with punctuation other than periods:

>>> input = "NLTK is a python library for working human-written text! Summarize is a package that uses NLTK to create summaries."
>>> ss.summarize(input, 1)
'NLTK is a python library for working human-written text!'

UPDATE

I've now (finally!) released this under the Apache 2.0 license, the same license as nltk, and put the module up on github (see above). Any contributions or suggestions are welcome.