Is it possible to create multiple instances of the same CNN that take in multiple images and are concatenated into a dense layer? (keras)

问题

Similar to this question, I'm looking to have several image input layers that go through one larger CNN (e.g. XCeption minus dense layers), and then have the output of the one CNN across all images be concatenated into a dense layer.

Is this possible with Keras or is it even possible to train a network from the ground-up with this architecture?

I'm essentially looking to train a model that takes in a larger but fixed number of images per sample (i.e. 3+ image inputs with similar visual features), but not to explode the number of parameters by training several CNNs at once. The idea is to train only one CNN, that can be used for all the outputs. Having all images go into the same dense layers is important so the model can learn the associations across multiple images, which are always ordered based on their source.

回答1:

You can easily achieve this using the Keras functional API the following way.

from tensorflow.python.keras import layers, models, applications

# Multiple inputs
in1 = layers.Input(shape=(128,128,3))
in2 = layers.Input(shape=(128,128,3))
in3 = layers.Input(shape=(128,128,3))

# CNN output
cnn = applications.xception.Xception(include_top=False)


out1 = cnn(in1)
out2 = cnn(in2)
out3 = cnn(in3)

# Flattening the output for the dense layer
fout1 = layers.Flatten()(out1)
fout2 = layers.Flatten()(out2)
fout3 = layers.Flatten()(out3)

# Getting the dense output
dense = layers.Dense(100, activation='softmax')

dout1 = dense(fout1)
dout2 = dense(fout2)
dout3 = dense(fout3)

# Concatenating the final output
out = layers.Concatenate(axis=-1)([dout1, dout2, dout3])

# Creating the model
model = models.Model(inputs=[in1,in2,in3], outputs=out)
model.summary()```

来源：https://stackoverflow.com/questions/58794981/is-it-possible-to-create-multiple-instances-of-the-same-cnn-that-take-in-multipl