What's the difference between convolutional and recurrent neural networks? [closed]

Answer 1

It is more helpful to describe the convolution and recurrent layers first.

Convolution layer:

Includes input, one or more filters (as well as subsampling).

The input can be one-dimensional or n-dimensional (n>1), for example, it can be a two-dimensional image. One or more filters are also defined in each layer. Inputs are convolving with each filter. The method of convolution is almost similar to the convolution of filters in image processing. In general, the purpose of this section is to extract the features of each filter from the input. The output of each convolution is called a feature map.

For example, a filter is considered for horizontal edges, and the result of its convolution with the input is the extraction of the horizontal edges of the input image. Usually, in practice and especially in the first layers, a large number of filters (for example, 60 filters in one layer) are defined. Also, after convolution, the subsampling operation is usually performed, for example, their maximum or average of each of the two neighborhood values ​​is selected.

The convolution layer allows important features and patterns to be extracted from the input. And delete input data dependencies (linear and nonlinear).

[The following figure shows an example of the use of convolutional layers and pattern extraction for classification.][1]

[1]: https://i.stack.imgur.com/HS4U0.png [Kalhor, A. (2020). Classification and Regression NNs. Lecture.]

Advantages of convolutional layers:

• Able to remove correlations and reduce input dimensions

• Network generalization is increasing

• Network robustness increases against changes because it extracts key features

• Very powerful and widely used in supervised learning

• ...

Recurrent layers:

In these layers, the output of the current layer or the output of the next layers can also be used as the input of the layer. It also can receive time series as input.

The output without using the recurrent layer is as follows (a simple example):

y = f(W * x)

Where x is input, W is weight and f is the activator function.

But in recurrent networks it can be as follows:

y = f(W * x)
y = f(W * y)
y = f(W * y)
... until convergence

This means that in these networks the generated output can be used as an input and thus have memory networks. Some types of recurrent networks are Discrete Hopfield Net and Recurrent Auto-Associative NET, which are simple networks or complex networks such as LSTM.

An example is shown in the image below.

Advantages of Recurrent Layers:

• They have memory capability

• They can use time series as input.

• They can use the generated output for later use.

• Very used in machine translation, voice recognition, image description

• ...

Networks that use convolutional layers are called convolutional networks (CNN). Similarly, networks that use recurrent layers are called recurrent networks. It is also possible to use both layers in a network according to the desired application!

Answer 2

First, we need to know that recursive NN is different from recurrent NN. By wiki's definition,

A recursive neural network (RNN) is a kind of deep neural network created by applying the same set of weights recursively over a structure

In this sense, CNN is a type of Recursive NN. On the other hand, recurrent NN is a type of recursive NN based on time difference. Therefore, in my opinion, CNN and recurrent NN are different but both are derived from recursive NN.

Answer 3

These architectures are completely different, so it is rather hard to say "what is the difference", as the only thing in common is the fact, that they are both neural networks.

Convolutional networks are networks with overlapping "reception fields" performing convolution tasks.

Recurrent networks are networks with recurrent connections (going in the opposite direction of the "normal" signal flow) which form cycles in the network's topology.

Answer 4

Convolutional neural networks (CNN) are designed to recognize images. It has convolutions inside, which see the edges of an object recognized on the image. Recurrent neural networks (RNN) are designed to recognize sequences, for example, a speech signal or a text. The recurrent network has cycles inside that implies the presence of short memory in the net. We have applied CNN as well as RNN choosing an appropriate machine learning algorithm to classify EEG signals for BCI: http://rnd.azoft.com/classification-eeg-signals-brain-computer-interface/

Answer 5

Apart from others, in CNN we generally use a 2d squared sliding window along an axis and convolute (with original input 2d image) to identify patterns.

In RNN we use previously calculated memory. If you are interested you can see, LSTM (Long Short-Term Memory) which is a special kind of RNN.

Both CNN and RNN have one point in common, as they detect patterns and sequences, that is you can't shuffle your single input data bits.

Answer 6

Convolutional neural networks (CNNs) for computer vision, and recurrent neural networks (RNNs) for natural language processing.

Although this can be applied in other areas, RNNs have the advantage of networks that can have signals travelling in both directions by introducing loops in the network.

Feedback networks are powerful and can get extremely complicated. Computations derived from the previous input are fed back into the network, which gives them a kind of memory. Feedback networks are dynamic: their state is changing continuously until they reach an equilibrium point.