Keras backend mean function: “ 'float' object has no attribute 'dtype' ”?

问题

I am trying to introduce a new kernel regularize for a network using Keras. But, it gives me the error: 'float' object has no attribute 'dtype' How can I fix it?

I found the code here: KL divergence in keras(tensorflow backend)

Here is my code:

    from keras import backend as K

    kullback_leibler_divergence = keras.losses.kullback_leibler_divergence

    def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
                      + kullback_leibler_divergence(1 - 0.05, 1 - means)))


    model = Sequential([
        Dense(100, input_shape=(x_train_s.shape[1],),kernel_initializer='random_uniform'),
        Activation('elu'),
        Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
        Activation('tanh')
    ])

    model.compile(optimizer='adam',loss='mean_squared_error')

    model.fit(x_train_s, x_train_s, epochs=5,validation_split=0.1, shuffle=True, verbose=1,batch_size=np.uint(x_train_s.shape[0]/100))

The full error trace back is presented here:

AttributeError                            Traceback (most recent call last)
<ipython-input-39-59bc90c687de> in <module>
     39     Activation('elu'),
     40     Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
---> 41     Activation('tanh')
     42 ])
     43 

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in __init__(self, layers, name)
     91         if layers:
     92             for layer in layers:
---> 93                 self.add(layer)
     94 
     95     @property

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in add(self, layer)
    179                 self.inputs = network.get_source_inputs(self.outputs[0])
    180         elif self.outputs:
--> 181             output_tensor = layer(self.outputs[0])
    182             if isinstance(output_tensor, list):
    183                 raise TypeError('All layers in a Sequential model '

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
    429                                          'You can build it manually via: '
    430                                          '`layer.build(batch_input_shape)`')
--> 431                 self.build(unpack_singleton(input_shapes))
    432                 self.built = True
    433 

C:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core.py in build(self, input_shape)
    864                                       name='kernel',
    865                                       regularizer=self.kernel_regularizer,
--> 866                                       constraint=self.kernel_constraint)
    867         if self.use_bias:
    868             self.bias = self.add_weight(shape=(self.units,),

C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    253         if regularizer is not None:
    254             with K.name_scope('weight_regularizer'):
--> 255                 self.add_loss(regularizer(weight))
    256         if trainable:
    257             self._trainable_weights.append(weight)

<ipython-input-39-59bc90c687de> in kl_divergence_regularizer(inputs)
     14     means = K.mean((inputs))
     15 #     means=1e-6
---> 16     return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
     17                   + kullback_leibler_divergence(1 - 0.05, 1 - means)))
     18 #     return 1e-10

C:\ProgramData\Anaconda3\lib\site-packages\keras\losses.py in kullback_leibler_divergence(y_true, y_pred)
     79 
     80 def kullback_leibler_divergence(y_true, y_pred):
---> 81     y_true = K.clip(y_true, K.epsilon(), 1)
     82     y_pred = K.clip(y_pred, K.epsilon(), 1)
     83     return K.sum(y_true * K.log(y_true / y_pred), axis=-1)

C:\ProgramData\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in clip(x, min_value, max_value)
   1599     if max_value is None:
   1600         max_value = np.inf
-> 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)
   1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)
   1603     return tf.clip_by_value(x, min_value, max_value)

AttributeError: 'float' object has no attribute 'dtype'

I need the KL divergence between 0.05 and mean calculate the following sum over i:

KL=sum(0.05*\log(0.05/mean[i]))

回答1:

You should change your KL divergence regularizer to:

def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        down = 0.05 * K.ones_like(means)
        up = (1 - 0.05) * K.ones_like(means)
        return 0.5 *(0.01 * (kullback_leibler_divergence(down, means)
                      + kullback_leibler_divergence(up, 1 - means)))

This is because the KL divergence takes two vectors/tensors, the KL between a scalar and a vector cannot be computed (its not defined).

回答2:

kullback_leibler_divergence(0.05, means)

This is a loss function. It is expecting y_true, y_pred, both as tensors. You are passing a float (0.05), and the system is trying to get the properties of this supposed tensor, but it's not a tensor.

The same problem will happen in the second kullback_leibler_divergence call where you pass again a float 1 - 0.05 instead of a tensor.

A simple solution (but you must check whether this is mathematically reasonable -- I don't know what KL_diff is supposed to do) is use a K.ones_like(means) * 0,05 and in the second call K.ones_like(means)*(1-0.05).

来源：https://stackoverflow.com/questions/57530823/keras-backend-mean-function-float-object-has-no-attribute-dtype