I am interested in the Cholesky decomposition of large sparse matrices. The problem I\'m having is that the Cholesky factors are not necessarily sparse (just like the produc
The problem of finding an optimal permutation of rows and columns of a matrix for a minimum fill-in matrix-factorization is not a trivial trask (as pointed out in the comments). Thus, heuristic algorithms are used in praxis.
There are some libraries that implement heuristic renumbering/ordering-strategies, often based on graph-algorithms for the adjacency-graph of your matrix. One tries to reduce the bandwidth of the corresponding adjacency-matrix. An easy to implement algroithms is the Cuthill-McKee Algorithm or the Minimum-Degree Ordering algorithm. More about this problem can be found in the Book Yousef Saad: Iterative Methods for Sparse Linear Systems (2003), upon many others.
Many libraries implement heuristic algorithms, like
Some of these libraries provide also sparse Cholesky factorization methods and can be used directly.