Why is the output of inv() and pinv() not equal in Matlab and Octave?

Question

I have noticed that if A is a NxN matrix and it has the inverse matrix. But what the inv() and pinv() function output is different. - My environment is Win7x64 SP1, Matlab

Answer 1

Floating point arithmetic has a certain precision, you can not rely on equality. To avoid such errors, you could try to work with the symbolic toolbox of matlab.

Very simple line of code in octave to demonstrate the problems:

>>> (1/48)*48==(1/49)*49
ans = 0
>>> (1/48)*48-(1/49)*49
ans =  1.1102e-16
>>>

Answer 2

This question is quite old, but I'll answer it anyway because it appears almost on top in some google searches.

I'll use for my example the magic(N) function which returns an N-by-N magic square.

I'll create a 3x3 magic square M3, take the pseudoinverse PI_M3 and multiply them:

   prompt_$ M3 = magic(3) , PI_M3 = pinv(M3) , M3 * PI_M3

  M3 =

    8   1   6
    3   5   7
    4   9   2

  PI_M3 =

     0.147222  -0.144444   0.063889
    -0.061111   0.022222   0.105556
    -0.019444   0.188889  -0.102778

  ans =

     1.0000e+00  -1.2212e-14   6.3283e-15
     5.5511e-17   1.0000e+00  -2.2204e-16
    -5.9952e-15   1.2268e-14   1.0000e+00

As you can see the answer is the identity matrix save for some rounding errors. I'll repeat the operation with a 4x4 magic square:

   prompt_$ M4 = magic(4) , PI_M4 = pinv(M4) , M4 * PI_M4

 
  M4 =

     16    2    3   13
      5   11   10    8
      9    7    6   12
      4   14   15    1

  PI_M4 =

     0.1011029  -0.0738971  -0.0613971   0.0636029
    -0.0363971   0.0386029   0.0261029   0.0011029
     0.0136029  -0.0113971  -0.0238971   0.0511029
    -0.0488971   0.0761029   0.0886029  -0.0863971

  ans =

     0.950000  -0.150000   0.150000   0.050000
    -0.150000   0.550000   0.450000   0.150000
     0.150000   0.450000   0.550000  -0.150000
     0.050000   0.150000  -0.150000   0.950000

The result is not the identity matrix, it means that the 4x4 magic square does not have an inverse. I can verify this by trying one of the rules of the Moore-Penrose pseudoinverse:

   prompt_$ M4 * PI_M4 * M4

  
ans =

   16.00000    2.00000    3.00000   13.00000
    5.00000   11.00000   10.00000    8.00000
    9.00000    7.00000    6.00000   12.00000
    4.00000   14.00000   15.00000    1.00000

The rule A*B*A = A is satisfied. This shows that pinv returns the inverse matrix when it is available and the pseudoinverse when the inverse is not available. This is the reason why in some situations you get a small difference, just some rounding errors, and in other situations you get a bigger difference. To show it I'll get the inverse of both magic quadrants and subtract them from the pseudoinverse:

   prompt_$ I_M3 = inv(M3) , I_M4 = inv(M4) , DIFF_M3 = PI_M3 - I_M3, DIFF_M4 = PI_M4 - I_M4

  I_M3 =

     0.147222  -0.144444   0.063889
    -0.061111   0.022222   0.105556
    -0.019444   0.188889  -0.102778

  warning: inverse: matrix singular to machine precision, rcond = 1.30614e-17
  I_M4 =

     9.3825e+13   2.8147e+14  -2.8147e+14  -9.3825e+13
     2.8147e+14   8.4442e+14  -8.4442e+14  -2.8147e+14
    -2.8147e+14  -8.4442e+14   8.4442e+14   2.8147e+14
    -9.3825e+13  -2.8147e+14   2.8147e+14   9.3825e+13

  DIFF_M3 =

     4.7184e-16  -1.0270e-15   5.5511e-16
    -9.9226e-16   2.0470e-15  -1.0825e-15
     5.2042e-16  -1.0270e-15   4.9960e-16

  DIFF_M4 =

    -9.3825e+13  -2.8147e+14   2.8147e+14   9.3825e+13
    -2.8147e+14  -8.4442e+14   8.4442e+14   2.8147e+14
     2.8147e+14   8.4442e+14  -8.4442e+14  -2.8147e+14
     9.3825e+13   2.8147e+14  -2.8147e+14  -9.3825e+13

Answer 3

It seems to me like you answered your own question in the bottom here. The reason is floating point arithmetic. The algortihms for inv() and pinv() are not exactly the same, as pinv() must be able to handle non-square matrices. Therefore the answers will not be exactly the same.

If you look at the value of pinv(A)*A, you will see that it's very very close to the identity matrix.

I get:

ans =

   1.0000e+00   6.1062e-16  -3.0809e-15
  -5.8877e-15   1.0000e+00   6.3942e-15
   2.4425e-15  -3.0184e-16   1.0000e+00

Instead of comparing the matrices with ==, use < tolerance_limit

c = A*pinv(A);
d = pinv(A)*A;

(c-d) < 1e-10

Sidenote:

x = A^-1*b should not be solved x = inv(A)*b;, but rather x = A \ b; See the link Shai posted for explanations.