Support Vector Machine or Artificial Neural Network for text processing?

Question

We need to decide between Support Vector Machines and Fast Artificial Neural Network for some text processing project.

It includes Contextual Spelling Correction and the

Answer 1

This question is very old. Lot of developments were happened in NLP area in last 7 years.

Convolutional_neural_network and Recurrent_neural_network evolved during this time.

Word Embeddings: Words appearing within similar context possess similar meaning. Word embeddings are pre-trained on a task where the objective is to predict a word based on its context.

CNN for NLP:

1. Sentences are first tokenized into words, which are further transformed into a word embedding matrix (i.e., input embedding layer) of d dimension.

2. Convolutional filters are applied on this input embedding layer to produce a feature map.

3. A max-pooling operation on each filter obtain a fixed length output and reduce the dimensionality of the output.

Since CNN had a short-coming of not preserving long-distance contextual information, RNNs have been introduced.

RNNs are specialized neural-based approaches that are effective at processing sequential information.

RNN memorizes the result of previous computations and use it in current computation.

There are few variations in RNN - Long Short Term Memory Unit (LSTM) and Gated recurrent units (GRUs)

Have a look at below resources:

deep-learning-for-nlp

Recent trends in deep learning paper