How to manipulate (constrain) the weights of the filter kernel in Conv2D in Keras?

问题

I understand that there are several options for kernel_constraint in Conv2D in Keras: max_norm, non_neg or unit_norm..

But what I needed is to set the anchor (center) position in the filter kernels to be zero. For example, if we have a filter kernel with its size of (width, height) = (5, 5), and that we have 3 channels in the input. I need to constrain the anchor (center) point of this kernel for every channel to be 0, like w(2,2,:)=0, assuming we put the channel dimension the 3rd dimension. If there are multiple filters, the anchor position of each filter should be with zero. How could I implement this?

I assume a custom kernel constraint is needed. This link gives suggest how to create one class inheriting from Constraint: https://github.com/keras-team/keras/issues/8196. This shows how in-built constraints are implemented: https://github.com/keras-team/keras/blob/master/keras/constraints.py

But still, I do not know how the dimensions of w are manipulated, and how to set the desired position to be zero. Any help is appreciated. Thanks.

Update: Daniel Möller's answer was tried. The error message is as follows:
raise ValueError('An operation has None for gradient. ' ValueError: An operation has None for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

Since Daniel can run that without problem on his side, to inspect what goes wrong in my program, I post my simplified code here. My data have 8 channels, but it shouldn't matter how many you have.

from keras.layers import Input, Conv2D
from keras.models import Model, optimizers
import numpy as np
import tensorflow as tf
from keras import backend as K
from keras.callbacks import ModelCheckpoint


class ZeroCenterConv2D(Conv2D):
    def __init__(self, filters, kernel_size, **kwargs):
        super(ZeroCenterConv2D, self).__init__(filters, kernel_size, **kwargs)

    def call(self, inputs):
        assert self.kernel_size[0] % 2 == 1, "Error: the kernel size is an even number"
        assert self.kernel_size[1] % 2 == 1, "Error: the kernel size is an even number"

        centerX = (self.kernel_size[0] - 1) // 2
        centerY = (self.kernel_size[1] - 1) // 2

        kernel_mask = np.ones(self.kernel_size + (1, 1))
        kernel_mask[centerX, centerY] = 0
        kernel_mask = K.constant(kernel_mask)

        customKernel = self.kernel * kernel_mask

        outputs = K.conv2d(
            inputs,
            customKernel,
            strides=self.strides,
            padding=self.padding,
            data_format=self.data_format,
            dilation_rate=self.dilation_rate)

        if self.activation is not None:
            return self.activation(outputs)

        return outputs


size1 = 256
size2 = 256
input_img = Input(shape=(size1, size2, 8))
conv1 = ZeroCenterConv2D(8, (5, 5), padding='same', activation='relu')(input_img)
autoencoder = Model(input_img, conv1)
adam = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-8)
autoencoder.compile(optimizer=adam, loss='mean_squared_error')


import scipy.io
A = scipy.io.loadmat('data_train')
x_train = A['data']
x_train = np.reshape(x_train, (1, 256, 256, 8))


from keras.callbacks import TensorBoard

autoencoder.fit(x_train, x_train,
                epochs=5,
                batch_size=1,
                shuffle=False,
                validation_data=(x_train, x_train),
                callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])


decoded_imgs = autoencoder.predict(x_train)

When conv1 = ZeroCenterConv2D... was replaced by the conventional conv1 = Conv2D..., everything works.

The full error message:

Connected to pydev debugger (build 181.4668.75)
/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Traceback (most recent call last):
  File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1664, in <module>
    main()
  File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1658, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1068, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/allen/autotion/temptest", line 62, in <module>
    callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
  File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/engine/training.py", line 1682, in fit
    self._make_train_function()
  File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/engine/training.py", line 992, in _make_train_function
    loss=self.total_loss)
  File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/optimizers.py", line 445, in get_updates
    grads = self.get_gradients(loss, params)
  File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/optimizers.py", line 80, in get_gradients
    raise ValueError('An operation has `None` for gradient. '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

Process finished with exit code 1

A further update: adding "bias" part in the code in Daniel's answer (already done), problems solved!

回答1:

You need a custom Conv2D layer for that, where you change its call method to apply the zero at the center.

class ZeroCenterConv2D(Conv2D):
    def __init__(self, filters, kernel_size, **kwargs):
        super(ZeroCenterConv2D, self).__init__(filters, kernel_size, **kwargs)

    def call(self, inputs):
        assert self.kernel_size[0] % 2 == 1, "Error: the kernel size is an even number"
        assert self.kernel_size[1] % 2 == 1, "Error: the kernel size is an even number"

        centerX = (self.kernel_size[0] - 1) // 2
        centerY = (self.kernel_size[1] - 1) // 2

        kernel_mask = np.ones(self.kernel_size + (1, 1))
        kernel_mask[centerX, centerY] = 0
        kernel_mask = K.variable(kernel_mask)

        customKernel = self.kernel * kernel_mask

        outputs = K.conv2d(
            inputs,
            customKernel,
            strides=self.strides,
            padding=self.padding,
            data_format=self.data_format,
            dilation_rate=self.dilation_rate)

        if self.use_bias:
            outputs = K.bias_add(
                outputs,
                self.bias,
                data_format=self.data_format)

        if self.activation is not None:
            return self.activation(outputs)

        return outputs

This will not replace the actual weights, though, but the center ones will never be used.

When you use layer.get_weights() of model.get_weights(), you will see the center weights as they were initialized (not as zeros).

来源：https://stackoverflow.com/questions/50513505/how-to-manipulate-constrain-the-weights-of-the-filter-kernel-in-conv2d-in-kera

标签

python

tensorflow

filter

keras

constraints