WARNING:tensorflow:sample_weight modes were coerced from … to ['…']

Question

Training an image classifier using .fit_generator() or .fit() and passing a dictionary to class_weight= as an argument.

I never go

Answer 1

I believe this is a bug with tensorflow that will happen when you call model.compile() with default parameter sample_weight_mode=None and then call model.fit() with specified sample_weight or class_weight.

From the tensorflow repos:

• fit() eventually calls _process_training_inputs()
• _process_training_inputs() sets sample_weight_modes = [None] based on model.sample_weight_mode = None and then creates a DataAdapter with sample_weight_modes = [None]
• the DataAdapter calls broadcast_sample_weight_modes() with sample_weight_modes = [None] during initialization
• broadcast_sample_weight_modes() seems to expect sample_weight_modes = None but receives [None]
• it asserts that [None] is a different structure from sample_weight / class_weight, overwrites it back to None by fitting to the structure of sample_weight / class_weight and outputs a warning

Warning aside this has no effect on fit() as sample_weight_modes in the DataAdapter is set back to None.

Note that tensorflow documentation states that sample_weight must be a numpy-array. If you call fit() with sample_weight.tolist() instead, you will not get a warning but sample_weight is silently overwritten to None when _process_numpy_inputs() is called in preprocessing and receives an input of length greater than one.

Answer 2

instead of providing a dictionary

weights = {'0': 42.0, '1': 1.0}

i tried a list

weights = [42.0, 1.0]

and the warning disappeared.

Answer 3

This seems like a bogus message. I get the same warning message after upgrading to TensorFlow 2.1, but I do not use any class weights or sample weights at all. I do use a generator that returns a tuple like this:

return inputs, targets

And now I just changed it to the following to make the warning go away:

return inputs, targets, [None]

I don't know if this is relevant, but my model uses 3 inputs, so my inputs variable is actually a list of 3 numpy arrays. targets is just a single numpy array.

In any case, it's just a warning. The training works fine either way.

Edit for TensorFlow 2.2:

This bug seems to have been fixed in TensorFlow 2.2, which is great. However the fix above will fail in TF 2.2, because it will try to get the shape of the sample weights, which will obviously fail with AttributeError: 'NoneType' object has no attribute 'shape'. So undo the above fix when upgrading to 2.2.

Answer 4

I have taken your Gist and installed Tensorflow 2.0, instead of TFA and it worked without any such Warning.

Here is the Gist of the complete code. Code for installing the Tensorflow is shown below:

!pip install tensorflow==2.0

Screenshot of the successful execution is shown below:

Update: This bug is fixed in Tensorflow Version 2.2.