How to use least squares with weight matrix?

问题

I know how to solve A.X = B by least squares using Python:

Example:

A=[[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,0,0]]
B=[1,1,1,1,1]
X=numpy.linalg.lstsq(A, B)
print X[0]
# [  5.00000000e-01   5.00000000e-01  -1.66533454e-16  -1.11022302e-16]

But what about solving this same equation with a weight matrix not being Identity:

A.X = B (W)

Example:

A=[[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,0,0]]
B=[1,1,1,1,1]
W=[1,2,3,4,5]

回答1:

I don't know how you have defined your weights, but you could try this if appropriate:

import numpy as np
A=np.array([[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,0,0]])
B = np.array([1,1,1,1,1])
W = np.array([1,2,3,4,5])
Aw = A * np.sqrt(W[:,np.newaxis])
Bw = B * np.sqrt(W)
X = np.linalg.lstsq(Aw, Bw)

回答2:

I found another approach (using W as a diagonal matrix, and matricial products) :

A=[[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,0,0]]
B = [1,1,1,1,1]
W = [1,2,3,4,5]
W = np.sqrt(np.diag(W))
Aw = np.dot(W,A)
Bw = np.dot(B,W)
X = np.linalg.lstsq(Aw, Bw)

Same values and same results.

回答3:

scikit package offers weighted regression directly .. https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression.fit

import numpy as np
# generate random data
N = 25
xp = [-5.0, 5.0]
x = np.random.uniform(xp[0],xp[1],(N,1))
e = 2*np.random.randn(N,1)
y = 2*x+e
w = np.ones(N)

# make the 3rd one outlier
y[2] += 30.0
w[2] = 0.0

from sklearn.linear_model import LinearRegression
# fit WLS using sample_weights
WLS = LinearRegression()
WLS.fit(x, y, sample_weight=w)

from matplotlib import pyplot as plt
plt.plot(x,y, '.')
plt.plot(xp, xp*WLS.coef_[0])
plt.show()

来源：https://stackoverflow.com/questions/27128688/how-to-use-least-squares-with-weight-matrix

标签

python

numpy

matrix

least-squares