Is there a fast way to invert a matrix in Matlab?

Question

I have lots of large (around 5000 x 5000) matrices that I need to invert in Matlab. I actually need the inverse, so I can\'t use mldivide instead, which is a lot faster for

Answer 1

First suppose the eigen values are all 1. Let A be the Jordan canonical form of your matrix. Then you can compute A^{-1} using only matrix multiplication and addition by

A^{-1} = I + (I-A) + (I-A)^2 + ... + (I-A)^k

where k < dim(A). Why does this work? Because generating functions are awesome. Recall the expansion

(1-x)^{-1} = 1/(1-x) = 1 + x + x^2 + ...

This means that we can invert (1-x) using an infinite sum. You want to invert a matrix A, so you want to take

A = I - X

Solving for X gives X = I-A. Therefore by substitution, we have

A^{-1} = (I - (I-A))^{-1} = 1 + (I-A) + (I-A)^2 + ...

Here I've just used the identity matrix I in place of the number 1. Now we have the problem of convergence to deal with, but this isn't actually a problem. By the assumption that A is in Jordan form and has all eigen values equal to 1, we know that A is upper triangular with all 1s on the diagonal. Therefore I-A is upper triangular with all 0s on the diagonal. Therefore all eigen values of I-A are 0, so its characteristic polynomial is x^dim(A) and its minimal polynomial is x^{k+1} for some k < dim(A). Since a matrix satisfies its minimal (and characteristic) polynomial, this means that (I-A)^{k+1} = 0. Therefore the above series is finite, with the largest nonzero term being (I-A)^k. So it converges.

Now, for the general case, put your matrix into Jordan form, so that you have a block triangular matrix, e.g.:

A 0 0
0 B 0
0 0 C

Where each block has a single value along the diagonal. If that value is a for A, then use the above trick to invert 1/a * A, and then multiply the a back through. Since the full matrix is block triangular the inverse will be

A^{-1} 0      0
0      B^{-1} 0
0      0      C^{-1}

There is nothing special about having three blocks, so this works no matter how many you have.

Note that this trick works whenever you have a matrix in Jordan form. The computation of the inverse in this case will be very fast in Matlab because it only involves matrix multiplication, and you can even use tricks to speed that up since you only need powers of a single matrix. This may not help you, though, if it's really costly to get the matrix into Jordan form.