How to scale the gradient during batch update in keras?

寵の児 提交于 2020-01-15 06:42:09

问题


I am using a standard keras model and I am training on batch (using the train_on_batch function). Now, I want to take the gradient of each element in the batch and scale it (multiply each sample gradient with a sample-specific value that I have) and after each gradient has been scaled, then it can be summed and used to update the existing weights. Is there anyway to do this given keras functions? And if not, is there a way for me to manipulate this using tensorflow? (given the model and the rest was written in keras)

The function looks like this: (the loop is to illustrate it happens for all samples in the batch)

grad = 0, w= #array of size batch_size
for i in batch_size:
    grad <- grad + w_i*grad_i

回答1:


  • Use the sample_weights argument in the fit method of a model.
  • Or, if using a generator, make the generator return not only X_train, y_train, but X_train, y_train, sample_weights.

In both cases, sample_weights should be a 1D vector with the same number of samples as the data.



来源:https://stackoverflow.com/questions/53661344/how-to-scale-the-gradient-during-batch-update-in-keras

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!