Keras - Fusion of a Dense Layer with a Convolution2D Layer

只愿长相守 提交于 2020-01-12 08:26:31

问题


I want to make a custom layer which is supposed to fuse the output of a Dense Layer with a Convolution2D Layer.

The Idea came from this paper and here's the network:

the fusion layer tries to fuse the Convolution2D tensor (256x28x28) with the Dense tensor (256). here's the equation for it:

y_global => Dense layer output with shape 256 y_mid => Convolution2D layer output with shape 256x28x28

Here's the description of the paper about the Fusion process:

I ended up making a new custom layer like below:

class FusionLayer(Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(FusionLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        input_dim = input_shape[1][1]
        initial_weight_value = np.random.random((input_dim, self.output_dim))
        self.W = K.variable(initial_weight_value)
        self.b = K.zeros((input_dim,))
        self.trainable_weights = [self.W, self.b]

    def call(self, inputs, mask=None):
        y_global = inputs[0]
        y_mid = inputs[1]
        # the code below should be modified
        output = K.dot(K.concatenate([y_global, y_mid]), self.W)
        output += self.b
        return self.activation(output)

    def get_output_shape_for(self, input_shape):
        assert input_shape and len(input_shape) == 2
        return (input_shape[0], self.output_dim)

I think I got the __init__ and build methods right but I don't know how to concatenate y_global (256 dimesnions) with y-mid (256x28x28 dimensions) in the call layer so that the output would be the same as the equation mentioned above.

How can I implement this equation in the call method?

Thanks so much...

UPDATE: any other way to successfully integrate the data of these 2 layers is also acceptable for me... it doesn't exactly have to be the way mentioned in the paper but it needs to at least return an acceptable output...


回答1:


I had to ask this question on the Keras Github page and someone helped me on how to implement it properly... here's the issue on github...




回答2:


In my opinion implementing a new kind of layer is a way to complicated for this task. I strongly advise you to use the following layers:

  • Flatten,
  • Merge,
  • Dense,

in order to obtain the expected behaviour.



来源:https://stackoverflow.com/questions/40810619/keras-fusion-of-a-dense-layer-with-a-convolution2d-layer

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!