How to build a multi-class convolutional neural network with Keras

三世轮回 提交于 2019-12-02 03:08:54

In my experience, also with a U-net for segmentation. It tends to do this:

  • Go to totally black or totally white
  • After a lot of time in which the loss seems to be frozen, it finds it's way.

I also use the "train just one image" method to find that convergence, then adding the other images is ok.

But I had to try a lot of times, and the only time it worked pretty fast was when I used:

  • final activation = 'sigmoid'
  • loss = 'binary_crossentropy'

But I wasn't using "relu" anywhere...perhaps that influences a little the convergence speed...? Thinking about "relu", which has only 0 or positive results, there is a big region in this function that does not have a gradient. Maybe having lots of "relu" activations creates a lot of "flat" areas without gradients? (Must think better about it to confirm)

Try a few times (and have patience to wait for many many epochs) with different weight initializations.

There is a chance that your learning rate is too big too.


About to_categorical(): have you tried to plot/print your masks? Do they really seem like what you expect them to?

Thank you @Daniel, your suggestions helped me in the end to get the Unet to work. I managed to get results that did not just classify the whole image as background when running 500+ epochs. Also, instead of using kernel_initializer='he_normal', kernel_initializer='zeros'or kernel_initializer=TruncatedNormal(mean=0.0, stddev=0.07) worked for me. I used 'sigmoid' activation function and loss='binary_crossentropy'. I kept the 'relu' activation for all the hidden convolutional layers. I noticed that my network will sometimes be stuck in a local minimum where the loss does not improve anymore, so I need to restart.

I don't see your prediction layer which as far as I know must be a dense layer and not a convolutional layer. Maybe that's your problem.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!