Solving an underdetermined scipy.sparse matrix using svd

Question

Problem

I have a set of equations with variables denoted with lowercase variables and constants with uppercase variables as such

A = a + b  
B = c + d          


        

Answer 1

The system you are solving has the form

[ 1 1 0 0 0 ] [a]   [A]
[ 0 0 1 1 0 ] [b] = [B]
[ 1 1 1 1 1 ] [c]   [C]
              [d]
              [e]

i.e., three equations for five variables a, b, c, d, e. As the answer linked in your question mentions, one can tackle such underdetermined system with the pseudoinverse, which Numpy directly provides in terms of the pinv function.

Since M has linearly independent rows, the psudoinverse has in this case the property that M.pinv(M) = I, where I denotes identity matrix (3x3 in this case). Thus formally, we can write the solution as:

v = pinv(M) . b

where v is the 5-component solution vector, and b denotes the right-hand side 3-component vector [A, B, C]. However, this solution is not unique, since one can add a vector from the so-called kernel or null space of the matrix M (i.e., a vector w for which M.w=0) and it will be still a solution:

M.(v + w) = M.v + M.w = b + 0 = b

Therefore, the only variables for which there is a unique solution are those for which the corresponding component of all possible vectors from the null space of M is zero. In other words, if you assemble the basis of the null space into a matrix (one basis vector per column), then the "solvable variables" will correspond to zero rows of this matrix (the corresponding component of any linear combination of the columns will be then also zero).

Let's apply this to your particular example:

import numpy as np
from numpy.linalg import pinv

M = [
    [1, 1, 0, 0, 0],
    [0, 0, 1, 1, 0],
    [1, 1, 1, 1, 1]
]

print(pinv(M))

[[ 5.00000000e-01 -2.01966890e-16  1.54302378e-16]
 [ 5.00000000e-01  1.48779676e-16 -2.10806254e-16]
 [-8.76351626e-17  5.00000000e-01  8.66819360e-17]
 [-2.60659800e-17  5.00000000e-01  3.43000417e-17]
 [-1.00000000e+00 -1.00000000e+00  1.00000000e+00]]

From this pseudoinverse, we see that the variable e (last row) is indeed expressible as - A - B + C. However, it also "predicts" that a=A/2 and b=A/2. To eliminate these non-unique solutions (equally valid would be also a=A and b=0 for example), let's calculate the null space borrowing the function from SciPy Cookbook:

print(nullspace(M))

[[ 5.00000000e-01 -5.00000000e-01]
 [-5.00000000e-01  5.00000000e-01]
 [-5.00000000e-01 -5.00000000e-01]
 [ 5.00000000e-01  5.00000000e-01]
 [-1.77302319e-16  2.22044605e-16]]

This function returns already the basis of the null space assembled into a matrix (one vector per column) and we see that, within a reasonable precision, the only zero row is indeed only the last one corresponding to the variable e.

EDIT:

For the set of equations

A = a + b, B = b + c, C = a + c

the corresponding matrix M is

[ 1 1 0 ]
[ 0 1 1 ]
[ 1 0 1 ]

Here we see that the matrix is in fact square, and invertible (the determinant is 2). Thus the pseudoinverse coincides with "normal" inverse:

[[ 0.5 -0.5  0.5]
 [ 0.5  0.5 -0.5]
 [-0.5  0.5  0.5]]

which corresponds to the solution a = (A - B + C)/2, .... Since M is invertible, its kernel / null space is empty, that's why the cookbook function returns only []. To see this, let's use the definition of the kernel - it is formed by all non-zero vectors x such that M.x = 0. However, since M^{-1} exists, x is given as x = M^{-1} . 0 = 0 which is a contradiction. Formally, this means that the found solution is unique (or that all variables are "solvable").

Answer 2

To build on ewcz's answer, both the nullspace and pseudo-inverse can be calculated using numpy.linalg.svd. See the links below:

pseudo-inverse

nullspace