Extracting last layers of keras model as a submodel

Question

Say we have a convolutional neural network M. I can extract features from images by using

extractor = Model(M.inputs, M.get_layer(\'last_conv\').output)
feat         


        

Answer 1

It depends on what you want to do.

• If you are going to throw away the feature extractor afterwards
• If you plan on training the feature extractor later

If you are going to use the extracted features but you don't intend on training the model used to generate them, you could use the predict method to get the features as you did:

features = extractor.predict(X)

then save its output to a file (np.save or cPickle or whatever). After that you could use that new dataset as the input to a new model.

If you plan on training the feature extractor later you'll need to stack the two networks as seen here with vgg as feature extractor https://github.com/fchollet/keras/issues/4576:

img_width, img_height = 150, 150
vgg16_model = VGG16(include_top=False, weights='imagenet')

input = Input(batch_shape=vgg16_model.output_shape)
x = GlobalAveragePooling2D()(input)
x = Dense(256, activation='relu')(x)
x = Dropout(0.5)(x)
predict = Dense(1, activation='sigmoid')(x)
top_model = Model(input, predict)

top_model.load_weights(os.path.join(data_path, 'VGG16Classifier.hdf5'))  

input = Input(shape=(3, img_width, img_height))
x = vgg16_model(input)
predict = top_model(x)
model = Model(input, predict)

PS: This example uses channels first ordering. If you are using tensorflow you should change the shape to shape=(img_width, img_height,3 )

Answer 2

This is not the nicest solution, but it works:

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dense, Dropout, Flatten

def cnn():
    model = Sequential()
    model.add(Conv2D(32, kernel_size=(3, 3),
                     activation='relu',
                     input_shape=(28, 28, 1), name='l_01'))
    model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
    model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
    model.add(Dropout(0.25, name='l_04'))
    model.add(Flatten(name='l_05'))
    model.add(Dense(128, activation='relu', name='l_06'))
    model.add(Dropout(0.5, name='l_07'))
    model.add(Dense(10, activation='softmax', name='l_08'))
    return model

def predictor(input_shape):
    model = Sequential()
    model.add(Flatten(name='l_05', input_shape=(12, 12, 64)))
    model.add(Dense(128, activation='relu', name='l_06'))
    model.add(Dropout(0.5, name='l_07'))
    model.add(Dense(10, activation='softmax', name='l_08'))
    return model

cnn_model = cnn()
cnn_model.save('/tmp/cnn_model.h5')

predictor_model = predictor(cnn_model.output.shape)
predictor_model.load_weights('/tmp/cnn_model.h5', by_name=True)

Answer 3

Every layer in the model is indexed. So if you know which layers you need, you could loop through them, copying them into a new model. This operation should copy the weights inside the layer as well.

Here's a model (from Oli Blum's answer):

  model = Sequential()
  # add some layers
  model.add(Conv2D(32, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=(28, 28, 1), name='l_01'))
  model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
  model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
  model.add(Dropout(0.25, name='l_04'))
  model.add(Flatten(name='l_05'))
  model.add(Dense(128, activation='relu', name='l_06'))
  model.add(Dropout(0.5, name='l_07'))
  model.add(Dense(10, activation='softmax', name='l_08'))

Say you wanted the last three layers:

def extract_layers(main_model, starting_layer_ix, ending_layer_ix):
  # create an empty model
  new_model = Sequential()
  for ix in range(starting_layer_ix, ending_layer_ix + 1):
    curr_layer = main_model.get_layer(index=ix)
    # copy this layer over to the new model
    new_model.add(curr_layer)
  return new_model