Image colorization using Keras - val_loss oscillates

Question

I have this sequential model:

conv2d (Conv2D)              (None, 256, 256, 32)      320
_________________________________________________________________
con