Tensorflow deep Q neural network wont learn with more than 1 hidden layer

Question

I\'ve been using tensorflow with a very simple deep Q neural network (NN) to play a very simple game.

The game is in an array [0, 0, 0, 0] and by picking one of the p