Reconstructing new data using sklearn NMF components Vs inverse_transform does not match

Question

I fit a model using scikit-learn NMF model on my training data. Now I perform an inverse transform of new data using

result_1 = model.inverse_transform(model.tr         


        

Answer 1

What happens

In scikit-learn, NMF does more than simple matrix multiplication: it optimizes!

Decoding (inverse_transform) is linear: the model calculates X_decoded = dot(W, H), where W is the encoded matrix, and H=model.components_ is a learned matrix of model parameters.

Encoding (transform), however, is nonlinear : it performs W = argmin(loss(X_original, H, W)) (with respect to W only), where loss is mean squared error between X_original and dot(W, H), plus some additional penalties (L1 and L2 norms of W), and with the constraint that W must be non-negative. Minimization is performed by coordinate descent, and result may be nonlinear in X_original. Thus, you cannot simply get W by multiplying matrices.

Why it is so weird

NMF has to perform such strange calculations because, otherwise, the model may produce negative results. Indeed, in your own example, you could try to perform transform by matrix multiplication

 print(np.dot(new_data, np.dot(model.components_.T, np.linalg.pinv(temp))))

and get the result W that contains negative numbers:

[[ 0.17328927  0.39649966]
 [ 0.1725572  -0.05780202]]

However, the coordinate descent within NMF avoids this problem by slightly modifying the matrix:

 print(model.transform(new_data))

gives a non-negative result

[[0.17328951 0.39649958]
 [0.16462405 0.        ]]

You can see that it does not simply clip W matrix from below, but modifies the positive elements as well, in order to improve the fit (and obey the regularization penalties).