Methods for automated synonym detection

Question

I am currently working on a neural network based approach to short document classification, and since the corpuses I am working with are usually around ten words, the standard s

Answer 1

There is an unsupervized boot-strapping approach that was explained to me to do this.

There are different ways of applying this approach, and variants, but here's a simplified version.

Concept:

Start by a assuming that if two words are synonyms, then in your corpus they will appear in similar settings. (eating grapes, eating sandwich, etc.)

(In this variant I will use co-occurence as the setting).

Boot-Strapping Algorithm:

We have two lists,

• one list will contain the words that co-occur with food items
• one list will contain the words that are food items

Supervized Part

Start by seeding one of the lists, for instance I might write the word Apple on the food items list.

Now let the computer take over.

Unsupervized Parts

It will first find all words in the corpus that appear just before Apple, and sort them in order of most occuring.

Take the top two (or however many you want) and add them into the co-occur with food items list. For example, perhaps "eating" and "Delicious" are the top two.

Now use that list to find the next two top food words by ranking the words that appear to the right of each word in the list.

Continue this process expanding each list until you are happy with the results.

Once that's done

(you may need to manually remove some things from the lists as you go which are clearly wrong.)

Variants

This procedure can be made quite effective if you take into account the grammatical setting of the keywords.

Subj ate NounPhrase
NounPhrase are/is Moldy

The workers harvested the Apples. 
   subj       verb     Apples 

That might imply harvested is an important verb for distinguishing foods.

Then look for other occurrences of subj harvested nounPhrase

You can expand this process to move words into categories, instead of a single category at each step.

My Source

This approach was used in a system developed at the University of Utah a few years back which was successful at compiling a decent list of weapon words, victim words, and place words by just looking at news articles.

An interesting approach, and had good results.

Not a neural network approach, but an intriguing methodology.

Edit:

the system at the University of Utah was called AutoSlog-TS, and a short slide about it can be seen here towards the end of the presentation. And a link to a paper about it here