How to build a multi-class convolutional neural network with Keras

Question

I am trying to implement a U-Net with Keras with Tensorflow backend for an image segmentation task. I have images of size (128,96) as input to the network together with mask ima

Answer 1

Thank you @Daniel, your suggestions helped me in the end to get the Unet to work. I managed to get results that did not just classify the whole image as background when running 500+ epochs. Also, instead of using kernel_initializer='he_normal', kernel_initializer='zeros'or kernel_initializer=TruncatedNormal(mean=0.0, stddev=0.07) worked for me. I used 'sigmoid' activation function and loss='binary_crossentropy'. I kept the 'relu' activation for all the hidden convolutional layers. I noticed that my network will sometimes be stuck in a local minimum where the loss does not improve anymore, so I need to restart.

Answer 2

In my experience, also with a U-net for segmentation. It tends to do this:

• Go to totally black or totally white
• After a lot of time in which the loss seems to be frozen, it finds it's way.

I also use the "train just one image" method to find that convergence, then adding the other images is ok.

But I had to try a lot of times, and the only time it worked pretty fast was when I used:

• final activation = 'sigmoid'
• loss = 'binary_crossentropy'

But I wasn't using "relu" anywhere...perhaps that influences a little the convergence speed...? Thinking about "relu", which has only 0 or positive results, there is a big region in this function that does not have a gradient. Maybe having lots of "relu" activations creates a lot of "flat" areas without gradients? (Must think better about it to confirm)

Try a few times (and have patience to wait for many many epochs) with different weight initializations.

There is a chance that your learning rate is too big too.

---

About to_categorical(): have you tried to plot/print your masks? Do they really seem like what you expect them to?

Answer 3

I don't see your prediction layer which as far as I know must be a dense layer and not a convolutional layer. Maybe that's your problem.