PyTorch - using `param.grad` affects learning process?

Question

I\'m training ResNet34 on CIFAR-10. I have a very weird behavior when I try to manipulate param.grad for model.parameters().

The following funct