How to find determinant of large matrix

Question

I found some C++ code for finding the determinant of matrix, for 4x4 to 8x8. It works ok, but my project needs matrices that are 18x18 or more, and the code is too slow. The

Answer 1

Well, not many of us working in the field would regard 18x18 as a large matrix and almost any technique you choose should be fast enough on any modern computer. Nor would many of us tackle matrix questions with recursive algorithms, much more likely to use iterative ones -- but that could be a reflection of the fact that a lot of people working on matrix problems are scientists and engineers not computer scientists.

I suggest you look at Numerical Recipes in C++. Not necessarily the best code you'll find, but it is a text for studying and learning from. For better codes, BOOST has a good reputation and there's always BLAS and things like the Intel Maths Kernel Library or the AMD Core Maths Library. I think all of these have implementations of determinant-finding routines which will tackle an 18x18 matrix very quickly.

Answer 2

The function det_recursive works for a square matrix of any size. However, since it is using recursive naive method of expanding Laplace's formula it is very slow for matrices of large size.

Another technique is to transform the matrix into an upper triangular form using gauss elimination technique. Then the determinant of the matrix is simply the products of diagonal elements of the triangular transformed form of the original matrix.

Basically numpy is the fastest but internally it uses some sort of linear matrix transformation method similar to what gauss elimination does. However, I am not sure what exactly it is!

In[1]
import numpy as np

In[2]
mat = np.random.rand(9,9)
print("numpy asnwer = ", np.linalg.det(mat))

Out[2] 
numpy asnwer =  0.016770106020608373

In[3]
def det_recursive(A):
    if A.shape[0] != A.shape[1]:
        raise ValueError('matrix {} is not Square'.format(A))

    sol = 0
    if A.shape != (1,1):
        for i in range(A.shape[0]):
            sol = sol +  (-1)**i * A[i, 0] * det_recursive(np.delete(np.delete(A, 0, axis= 1), i, axis= 0))
        return sol
    else:
        return A[0,0]
​

In[4]
print("recursive asnwer = ", det_recursive(mat))

Out[4]
recursive asnwer =  0.016770106020608397

In[5]
def det_gauss_elimination(a,tol=1.0e-9):
    """
    calculate determinant using gauss-elimination method
    """
    a = np.copy(a)

    assert(a.shape[0] == a.shape[1])
    n = a.shape[0]

    # Set up scale factors
    s = np.zeros(n)

    mult = 0
    for i in range(n):
        s[i] = max(np.abs(a[i,:])) # find the max of each row
    for k in range(0, n-1): #pivot row
        # Row interchange, if needed
        p = np.argmax(np.abs(a[k:n,k])/s[k:n]) + k 
        if abs(a[p,k]) < tol: 
            print("Matrix is singular")
            return 0
        if p != k: 
            a[[k,p],:] = a[[p, k],:] 
            s[k],s[p] = s[p],s[k]
            mult = mult + 1
​
        # convert a to upper triangular matrix
        for i in range(k+1,n):
            if a[i,k] != 0.0: # skip if a(i,k) is already zero
                lam = a [i,k]/a[k,k] 
                a[i,k:n] = a[i,k:n] - lam*a[k,k:n]
​
    deter = np.prod(np.diag(a))* (-1)**mult   
    return deter

In[6] 
print("gauss elimination asnwer = ", det_gauss_elimination(mat))

Out[6] 
gauss elimination asnwer =  0.016770106020608383

In[7] 
print("numpy time")
%timeit -n3 -r3 np.linalg.det(mat)
print("\nrecursion time")
%timeit -n3 -r3 det_recursive(mat)
print("\ngauss_elimination time")
%timeit -n3 -r3 det_gauss_elimination(mat)

Out[7]
numpy time
40.8 µs ± 17.2 µs per loop (mean ± std. dev. of 3 runs, 3 loops each)

recursion time
10.1 s ± 128 ms per loop (mean ± std. dev. of 3 runs, 3 loops each)

gauss_elimination time
472 µs ± 106 µs per loop (mean ± std. dev. of 3 runs, 3 loops each)

Answer 3

I assume you're using the naive method of expanding Laplace's formula. If you want to gain in speed, you can decompose your matrix M using LU decomposition (into two lower- and upper-diagonal matrices) which you can achieve with a modified Gauss-Jordan elimination in 2*n^3/3 FLOPS and then calculate the determinant as:

det(M) = det(L) * det(U), which for triangular matrices is just the product of the entries in their diagonal.

This process will still be faster than O(n!).

Edit: you can also use Crout's method, which is widely implemented.

Answer 4

I think this may work. I wrote it when I studied a numerical analysis course. This is not only the determinant but other functions related to matrices.

First, copy and save this code as Matrix.h:

//Title: Matrix Header File
//Writer: Say OL
//This is a beginner code not an expert one
//No responsibilty for any errors
//Use for your own risk

using namespace std;
int row,col,Row,Col;
double Coefficient;

//Input Matrix
void Input(double Matrix[9][9],int Row,int Col)
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
        {
            cout<<"e["<<row<<"]["<<col<<"]=";
            cin>>Matrix[row][col];
        }
}

//Output Matrix
void Output(double Matrix[9][9],int Row,int Col)
{
    for(row=1;row<=Row;row++)
    {
        for(col=1;col<=Col;col++)
            cout<<Matrix[row][col]<<"\t";
        cout<<endl;
    }
}

//Copy Pointer to Matrix
void CopyPointer(double (*Pointer)[9],double Matrix[9][9],int Row,int Col)
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            Matrix[row][col]=Pointer[row][col];
}

//Copy Matrix to Matrix
void CopyMatrix(double MatrixInput[9][9],double MatrixTarget[9][9],int Row,int Col)
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            MatrixTarget[row][col]=MatrixInput[row][col];
}

//Transpose of Matrix
double MatrixTran[9][9];
double (*(Transpose)(double MatrixInput[9][9],int Row,int Col))[9]
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            MatrixTran[col][row]=MatrixInput[row][col];
    return MatrixTran;
}

//Matrix Addition
double MatrixAdd[9][9];
double (*(Addition)(double MatrixA[9][9],double MatrixB[9][9],int Row,int Col))[9]
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            MatrixAdd[row][col]=MatrixA[row][col]+MatrixB[row][col];
    return MatrixAdd;
}

//Matrix Subtraction
double MatrixSub[9][9];
double (*(Subtraction)(double MatrixA[9][9],double MatrixB[9][9],int Row,int Col))[9]
{
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            MatrixSub[row][col]=MatrixA[row][col]-MatrixB[row][col];
    return MatrixSub;
}

//Matrix Multiplication
int mRow,nCol,pCol,kcol;
double MatrixMult[9][9];
double (*(Multiplication)(double MatrixA[9][9],double MatrixB[9][9],int mRow,int nCol,int pCol))[9]
{
    for(row=1;row<=mRow;row++)
        for(col=1;col<=pCol;col++)
        {
            MatrixMult[row][col]=0.0;
            for(kcol=1;kcol<=nCol;kcol++)
                MatrixMult[row][col]+=MatrixA[row][kcol]*MatrixB[kcol][col];
        }
    return MatrixMult;
}

//Interchange Two Rows
double RowTemp[9][9];
double MatrixInter[9][9];
double (*(InterchangeRow)(double MatrixInput[9][9],int Row,int Col,int iRow,int jRow))[9]
{
    CopyMatrix(MatrixInput,MatrixInter,Row,Col);
    for(col=1;col<=Col;col++)
    {
        RowTemp[iRow][col]=MatrixInter[iRow][col];
        MatrixInter[iRow][col]=MatrixInter[jRow][col];
        MatrixInter[jRow][col]=RowTemp[iRow][col];
    }
    return MatrixInter;
}

//Pivote Downward
double MatrixDown[9][9];
double (*(PivoteDown)(double MatrixInput[9][9],int Row,int Col,int tRow,int tCol))[9]
{
    CopyMatrix(MatrixInput,MatrixDown,Row,Col);
    Coefficient=MatrixDown[tRow][tCol];
    if(Coefficient!=1.0)
        for(col=1;col<=Col;col++)
            MatrixDown[tRow][col]/=Coefficient;
    if(tRow<Row)
        for(row=tRow+1;row<=Row;row++)
        {
            Coefficient=MatrixDown[row][tCol];
            for(col=1;col<=Col;col++)
                MatrixDown[row][col]-=Coefficient*MatrixDown[tRow][col];
        }
return MatrixDown;
}

//Pivote Upward
double MatrixUp[9][9];
double (*(PivoteUp)(double MatrixInput[9][9],int Row,int Col,int tRow,int tCol))[9]
{
    CopyMatrix(MatrixInput,MatrixUp,Row,Col);
    Coefficient=MatrixUp[tRow][tCol];
    if(Coefficient!=1.0)
        for(col=1;col<=Col;col++)
            MatrixUp[tRow][col]/=Coefficient;
    if(tRow>1)
        for(row=tRow-1;row>=1;row--)
        {
            Coefficient=MatrixUp[row][tCol];
            for(col=1;col<=Col;col++)
                MatrixUp[row][col]-=Coefficient*MatrixUp[tRow][col];
        }
    return MatrixUp;
}

//Pivote in Determinant
double MatrixPiv[9][9];
double (*(Pivote)(double MatrixInput[9][9],int Dim,int pTarget))[9]
{
    CopyMatrix(MatrixInput,MatrixPiv,Dim,Dim);
    for(row=pTarget+1;row<=Dim;row++)
    {
        Coefficient=MatrixPiv[row][pTarget]/MatrixPiv[pTarget][pTarget];
        for(col=1;col<=Dim;col++)
        {
            MatrixPiv[row][col]-=Coefficient*MatrixPiv[pTarget][col];
        }
    }
    return MatrixPiv;
}

//Determinant of Square Matrix
int dCounter,dRow;
double Det;
double MatrixDet[9][9];
double Determinant(double MatrixInput[9][9],int Dim)
{
    CopyMatrix(MatrixInput,MatrixDet,Dim,Dim);
    Det=1.0;
    if(Dim>1)
    {
        for(dRow=1;dRow<Dim;dRow++)
        {
            dCounter=dRow;
            while((MatrixDet[dRow][dRow]==0.0)&(dCounter<=Dim))
            {
                dCounter++;
                Det*=-1.0;
                CopyPointer(InterchangeRow(MatrixDet,Dim,Dim,dRow,dCounter),MatrixDet,Dim,Dim);
            }
            if(MatrixDet[dRow][dRow]==0)
            {
                Det=0.0;
                break;
            }
            else
            {
                Det*=MatrixDet[dRow][dRow];
                CopyPointer(Pivote(MatrixDet,Dim,dRow),MatrixDet,Dim,Dim);
            }
        }
        Det*=MatrixDet[Dim][Dim];
    }
    else Det=MatrixDet[1][1];
    return Det;
}

//Matrix Identity
double MatrixIdent[9][9];
double (*(Identity)(int Dim))[9]
{
    for(row=1;row<=Dim;row++)
        for(col=1;col<=Dim;col++)
            if(row==col)
                MatrixIdent[row][col]=1.0;
            else
                MatrixIdent[row][col]=0.0;
    return MatrixIdent;
}
//Join Matrix to be Augmented Matrix
double MatrixJoin[9][9];
double (*(JoinMatrix)(double MatrixA[9][9],double MatrixB[9][9],int Row,int ColA,int ColB))[9]
{
    Col=ColA+ColB;
    for(row=1;row<=Row;row++)
        for(col=1;col<=Col;col++)
            if(col<=ColA)
                MatrixJoin[row][col]=MatrixA[row][col];
            else
                MatrixJoin[row][col]=MatrixB[row][col-ColA];
    return MatrixJoin;
}

//Inverse of Matrix
double (*Pointer)[9];
double IdentMatrix[9][9];
int Counter;
double MatrixAug[9][9];
double MatrixInv[9][9];
double (*(Inverse)(double MatrixInput[9][9],int Dim))[9]
{
    Row=Dim;
    Col=Dim+Dim;
    Pointer=Identity(Dim);
    CopyPointer(Pointer,IdentMatrix,Dim,Dim);
    Pointer=JoinMatrix(MatrixInput,IdentMatrix,Dim,Dim,Dim);
    CopyPointer(Pointer,MatrixAug,Row,Col);
    for(Counter=1;Counter<=Dim;Counter++)   
    {
        Pointer=PivoteDown(MatrixAug,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixAug,Row,Col);
    }
    for(Counter=Dim;Counter>1;Counter--)
    {
        Pointer=PivoteUp(MatrixAug,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixAug,Row,Col);
    }
    for(row=1;row<=Dim;row++)
        for(col=1;col<=Dim;col++)
            MatrixInv[row][col]=MatrixAug[row][col+Dim];
    return MatrixInv;
}

//Gauss-Jordan Elemination
double MatrixGJ[9][9];
double VectorGJ[9][9];
double (*(GaussJordan)(double MatrixInput[9][9],double VectorInput[9][9],int Dim))[9]
{
    Row=Dim;
    Col=Dim+1;
    Pointer=JoinMatrix(MatrixInput,VectorInput,Dim,Dim,1);
    CopyPointer(Pointer,MatrixGJ,Row,Col);
    for(Counter=1;Counter<=Dim;Counter++)   
    {
        Pointer=PivoteDown(MatrixGJ,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixGJ,Row,Col);
    }
    for(Counter=Dim;Counter>1;Counter--)
    {
        Pointer=PivoteUp(MatrixGJ,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixGJ,Row,Col);
    }
    for(row=1;row<=Dim;row++)
        for(col=1;col<=1;col++)
            VectorGJ[row][col]=MatrixGJ[row][col+Dim];
    return VectorGJ;
}

//Generalized Gauss-Jordan Elemination
double MatrixGGJ[9][9];
double VectorGGJ[9][9];
double (*(GeneralizedGaussJordan)(double MatrixInput[9][9],double VectorInput[9][9],int Dim,int vCol))[9]
{
    Row=Dim;
    Col=Dim+vCol;
    Pointer=JoinMatrix(MatrixInput,VectorInput,Dim,Dim,vCol);
    CopyPointer(Pointer,MatrixGGJ,Row,Col);
    for(Counter=1;Counter<=Dim;Counter++)   
    {
        Pointer=PivoteDown(MatrixGGJ,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixGGJ,Row,Col);
    }
    for(Counter=Dim;Counter>1;Counter--)
    {
        Pointer=PivoteUp(MatrixGGJ,Row,Col,Counter,Counter);
        CopyPointer(Pointer,MatrixGGJ,Row,Col);
    }
    for(row=1;row<=Row;row++)
        for(col=1;col<=vCol;col++)
            VectorGGJ[row][col]=MatrixGGJ[row][col+Dim];
    return VectorGGJ;
}

//Matrix Sparse, Three Diagonal Non-Zero Elements
double MatrixSpa[9][9];
double (*(Sparse)(int Dimension,double FirstElement,double SecondElement,double ThirdElement))[9]
{
    MatrixSpa[1][1]=SecondElement;
    MatrixSpa[1][2]=ThirdElement;
    MatrixSpa[Dimension][Dimension-1]=FirstElement;
    MatrixSpa[Dimension][Dimension]=SecondElement;
    for(int Counter=2;Counter<Dimension;Counter++)
    {
        MatrixSpa[Counter][Counter-1]=FirstElement;
        MatrixSpa[Counter][Counter]=SecondElement;
        MatrixSpa[Counter][Counter+1]=ThirdElement;
    }
    return MatrixSpa;
}

In my method, I convert the matrix to upper triangular matrix using elementary row operations. And the determinant is the product of the diagonal elements.

Here is the sample code:

#include<iostream>
#include<conio.h>
#include"Matrix.h"

int Dim;
double Matrix[9][9];

int main()
{
    cout<<"Enter matrix dimension: ";
    cin>>Dim;
    cout<<"Enter matrix elements:"<<endl;
    Input(Matrix,Dim,Dim);
    cout<<"Your matrix:"<<endl;
    Output(Matrix,Dim,Dim);
    cout<<"The determinant: "<<Determinant(Matrix,Dim)<<endl;
    getch();
}

Answer 5

Since I can't comment, I wish to add this: the Cholesky decomposition (or its variant, LDL^T, L a unit lower triangular matrix and D a diagonal matrix) can be used to verify if a symmetric matrix is positive/negative definite: if it is positive definite, the elements of D are all positive, and the Cholesky decomposition will finish successfully without taking the square root of a negative number. If the matrix is negative definite, the elements of D are all negative, the matrix itself will not have a Cholesky decomposition, but the negative of it would.

"Calculating the determinant of a triangular matrix is simple: multiply the diagonal elements, as the cofactors of the off-diagonal terms are 0. Using an LU decomposition further simplifies this, as L is a unit, lower triangular matrix, i.e. its diagonal elements are all 1, in most implementations. Therefor, you often only have to calculate the determinant of U."

• You forgot here to take into account that all practical implementations of Gaussian elimination make use of (partial) pivoting for extra numerical stability; so your description is incomplete; one counts the number of row swaps done during the decomposition phase, and after multiplying together all the diagonal elements of U, this product should be negated if the number of swaps is odd.

As for code, NR is not free; I suggest taking a look at LAPACK/CLAPACK/LAPACK++ @ http://www.netlib.org/ instead. For reference, I can do no better than point you to "Matrix Computations" by Golub and Van Loan.