How to Identify Recurrent Connections in an Arbitrary Neural Network

Question

I am trying to implement Neuro-Evolution of Augmenting Topologies in C#. I am running into a problem with recurrent connections. I understand that, for a recurrent connectio

Answer 1

I had a similar problem when i started implememting this paper. I don't know what your network looks like in the momen, so i'll explain to you what i did.

My network starts out as input & output layers only. To create connections and neurons i implemented some kind of DNA (in my case this is an array of instructions like 'connect neuron nr. 2 with neuron nr. 5 and set the weight to 0.4'). Every neuron in my network has a "layerNumber" which tells me where a neuron is in my network. This layerNumber is set for every in and output neuron. for inputneurons i used Double.minvalue and for outputneurons i used Double.maxvalue.

This is the basic setup. From now on just follow these rules when modifying the network:

• Whenever you want to create a connection, make sure the 'from' neuron has a layerNumber < Double.maxValue

• Whenever you want to create a connection, make sure that the 'to' neuron has a bigger layerNumber than the 'from' neuron.

• Whenever a connection is split up into 2 connections and a neuron between them, set the neurons layerNumber to NeuronFrom.layerNumber*0.5 + NeuronTo.layerNumber*0.5 This is important, you can't add them and simply divide by 2, because this would likely result in Double.maxValue + something, which would return some weird number (i guess an overflow would happen, so you would get a negative number?).

If you follow all the rules you should always have forwarding connections only. No recurrent ones. If you want recurrent connections you can create them by just swapping 'from' & 'to' while creating a new connection.

Pro tricks: Use only one ArrayList of Neurons. Make the DNA use ID's of neurons to find them, but create a 'Connection' class which will have the Neuron objects as attributes. When filtering your connections/neurons use ArrayList.stream().filter()

When later propagating trough the network you can just sort your neurons by layerNumber, set the inputValues and go trough all neurons using a for() loop. Just calculate the neurons outputvalue and transfer it to every neuron which has a connection where 'from' is == the current neuron.

Hope it's not too complicated...