问题
Two simple questions about Tensorflow's loss and sample weights. Imagine I have shallow fully convolutional NN with next model: Image(16x16x1)->Conv2(16x16x10)->so output is vector o[1][1][10] with 10 neurons. Because of batch 32, we have final output tensor as [32][1][1][10] (all dimentions checked by myself carefully).
So now questions:
- I have experience of writing in C++ and understanding backpropagation, so I don't understand why, for example MSE loss in TF, use reduction of last dimention (see https://www.tensorflow.org/api_docs/python/tf/keras/losses/MSE):
loss = mean(square(y_true - y_pred), axis=-1)
So for my case of output tensor [32][1][1][10] after reduction will be formed tensor of [32][1][1] dimentions. But how it will backpropagate error back through layers for each element from last dimention with size of [10] if it has avarage them? So after averaging we got scalar, not vector. How it is possible from scalar to backpropagate error from [10] outputs? Please note that these elements from [10] cannot be averaged because of different meaning of each element (one of them could be length, other speed, etc).
Question about sample weights. Absolutely nothing of examples from internet. I have been triing to implement simple element wize multiplication output weighted loss = output tensor [32][1][1][10] element_wize to sample weights [32][1][1][10]. As I understand sample weights has to be the same dimentionality. My compile function model.compile(..., sample_weigh_mode='temporal' ,...) Element x for model.fit is going from generator. So generator gives al three tensors X_train, Y_train and sample_weights. All this is working very strange and I don't understand dynamics. Also it is strange that if I gives sample_weights tensor with not 10 outputs (may 1 or 100, so any) like as number of outputs of my NN, it still somehow works and not rise any errors or exceptions.
So help me please to make clear all this staff for me. Thanks, Alexey.
来源：https://stackoverflow.com/questions/64909383/tensorflow-loss-sample-weights