How to avoid overfitting on a simple feed forward network

Question

Using the pima indians diabetes dataset I\'m trying to build an accurate model using Keras. I\'ve written the following code:

# Visualize training history
from k         


        

Answer 1

First, try adding some regularization (https://keras.io/regularizers/) like with this code:

model.add(Dense(12, input_dim=12,
            kernel_regularizer=regularizers.l2(0.01),
            activity_regularizer=regularizers.l1(0.01)))

Also, make sure to decrease your network size i.e. you don't need a hidden layer of 500 neurons - try just taking that out to decrease the representation power and maybe even another layer if it's still overfitting. Also, only use relu activation. Maybe also try increasing your dropout rate to something like 0.75 (although it's already high). You probably also don't need to run it for so many epochs - it will just begin to overfit after long enough.