autograd

Partial Derivative using Autograd

由 天大地大妈咪最大 提交于 2020-01-01 06:55:07
问题 I have a function that takes in a multivariate argument x. Here x = [x1,x2,x3]. Let's say my function looks like: f(x,T) = np.dot(x,T) + np.exp(np.dot(x,T) where T is a constant. I am interested in finding df/dx1, df/dx2 and df/dx3 functions. I have achieved some success using scipy diff, but I am a bit skeptical because it uses numerical differences. Yesterday, my colleague pointed me to Autograd (github). Since it seems to be a popular package, I am hoping someone here knows how to get

Autograd.grad() for Tensor in pytorch

由 ﹥＞﹥吖頭↗ 提交于 2019-12-24 08:10:05
问题 I want to compute the gradient between two tensors in a net. The input X tensor (batch size x m) is sent through a set of convolutional layers which give me back and output Y tensor(batch size x n). I’m creating a new loss and I would like to know the gradient of Y w.r.t. X. Something that in tensorflow would be like: tf.gradients( ys =Y, xs = X) Unfortunately, I’ve been making tests with torch.autograd.grad(), but I could not figure out how to do it. I get errors like: “RunTimeerror: grad

PyTorch autograd — grad can be implicitly created only for scalar outputs

由 拜拜、爱过 提交于 2019-12-22 08:57:41
问题 I am using the autograd tool in PyTorch , and have found myself in a situation where I need to access the values in a 1D tensor by means of an integer index. Something like this: def basic_fun(x_cloned): res = [] for i in range(len(x)): res.append(x_cloned[i] * x_cloned[i]) print(res) return Variable(torch.FloatTensor(res)) def get_grad(inp, grad_var): A = basic_fun(inp) A.backward() return grad_var.grad x = Variable(torch.FloatTensor([1, 2, 3, 4, 5]), requires_grad=True) x_cloned = x.clone()

Pytorch - RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed

由 余生长醉 提交于 2019-12-20 17:47:10
问题 I keep running into this error: RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time. I had searched in Pytorch forum, but still can’t find out what I have done wrong in my custom loss function. My model is nn.GRU, and here is my custom loss function: def _loss(outputs, session, items): # `items` is a dict() contains embedding of all items def f(output, target): pos = torch

Pytorch. Can autograd be used when the final tensor has more than a single value in it?

由 雨燕双飞 提交于 2019-12-20 03:43:16
问题 Can autograd be used when the final tensor has more than a single value in it? I tried the following. x = torch.tensor([4.0, 5.0], requires_grad=True) y = x ** 2 print(y) y.backward() Throws an error RuntimeError: grad can be implicitly created only for scalar outputs The following however works. x = torch.tensor([4.0, 5.0], requires_grad=True) y = x ** 2 y = torch.sum(y) print(y) y.backward() print(x.grad) The output is as tensor(41., grad_fn=<SumBackward0>) tensor([ 8., 10.]) Am I missing

Mini batch training for inputs of variable sizes

由 核能气质少年 提交于 2019-12-11 02:22:17
问题 I have a list of LongTensors, and another list of labels. I'm new to PyTorch and RNN's so I'm quite confused as to how to implement minibatch training for the data I have. There is much more to this data, but I want to keep it simple, so I can understand only how to implement the minibatch training part. I'm doing multiclass classification based on the final hidden state of an LSTM/GRU trained on variable length inputs. I managed to get it working with batch size 1(basically SGD) but I'm

PyTorch autograd — grad can be implicitly created only for scalar outputs

由 被刻印的时光 ゝ 提交于 2019-12-05 18:01:37
I am using the autograd tool in PyTorch , and have found myself in a situation where I need to access the values in a 1D tensor by means of an integer index. Something like this: def basic_fun(x_cloned): res = [] for i in range(len(x)): res.append(x_cloned[i] * x_cloned[i]) print(res) return Variable(torch.FloatTensor(res)) def get_grad(inp, grad_var): A = basic_fun(inp) A.backward() return grad_var.grad x = Variable(torch.FloatTensor([1, 2, 3, 4, 5]), requires_grad=True) x_cloned = x.clone() print(get_grad(x_cloned, x)) I am getting the following error message: [tensor(1., grad_fn=

Pytorch - RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed

由 风流意气都作罢 提交于 2019-12-03 06:21:14
I keep running into this error: RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time. I had searched in Pytorch forum, but still can’t find out what I have done wrong in my custom loss function. My model is nn.GRU, and here is my custom loss function: def _loss(outputs, session, items): # `items` is a dict() contains embedding of all items def f(output, target): pos = torch.from_numpy(np.array([items[target["click"]]])).float() neg = torch.from_numpy(np.array([items[idx] for idx

Pytorch. Can autograd be used when the final tensor has more than a single value in it?

由 孤街浪徒 提交于 2019-12-02 00:58:53
Can autograd be used when the final tensor has more than a single value in it? I tried the following. x = torch.tensor([4.0, 5.0], requires_grad=True) y = x ** 2 print(y) y.backward() Throws an error RuntimeError: grad can be implicitly created only for scalar outputs The following however works. x = torch.tensor([4.0, 5.0], requires_grad=True) y = x ** 2 y = torch.sum(y) print(y) y.backward() print(x.grad) The output is as tensor(41., grad_fn=<SumBackward0>) tensor([ 8., 10.]) Am I missing something here or can I proceed with the assumption that autograd only works when the final tensor has a

backward function in PyTorch

由 浪子不回头ぞ 提交于 2019-12-01 08:10:04
i have some question about pytorch's backward function i don't think i'm getting the right output import numpy as np import torch from torch.autograd import Variable a = Variable(torch.FloatTensor([[1,2,3],[4,5,6]]), requires_grad=True) out = a * a out.backward(a) print(a.grad) the output is tensor([[ 2., 8., 18.], [32., 50., 72.]]) maybe it's 2*a*a but i think the output suppose to be tensor([[ 2., 4., 6.], [8., 10., 12.]]) 2*a. cause d(x^2)/dx=2x Please read carefully the documentation on backward() to better understand it. By default, pytorch expects backward() to be called for the last
订阅 autograd