Creating Huge Sparse Matrices in python

问题

I have been using normal matrices from numpy to store a Matrix for a physics project. The size of the matrix is determined by the physical system. So for instance if the system has parameters:
L=4 and N =2, then the matrix is of dimension 4C2 = 6, so the matrix is a 6x6 matrix.

This is fine except for now I need larger size i.e 20C10 = 184,756. So the matrix required is now a 184756x184756 matrix, which when I try to create an empty matrix of this size gives me a memory error. (with 16GB of RAM)

The resultant matrix is mostly just diagonal and off diagonal terms, so there are a huge amount of zeroes in the large size matrices. Hence Sparse Matrices seem like the correct approach.

I have tried to get it to work by looking at other answers and just trying by myself from the python libraries, but to no avail.

Below is the code for my normal matrix:

def HamGen(L,N,delta,J):
    """
    Will Generate the hamiltonian matrix, 
    Takes parameters:
        L : Number of sites
        N : Number of spin downs
        delta : anistropy 

    Each term is gotten by getting H(i,j) = <Set(i)|H|Set(j)>
    The term will be a number
    Where H is an operator that acts on elements of the set 
    """
    D = BS.dimension(L,N) # Gets the dimension of the matrix, i.e NxN matrix
    Hamiltonian = np.zeros((D,D)) # Creates empty matrix 

    count1 = 0
    Set = BS.getSet(L,N) # The set of states to construct the hamiltonian

    for alpha in Set: #loop through the set (i)
        count2 = 0
        for beta in Set: # j
            """
            Compute ab = <alpha|Hamiltonian|beta>
            Then let Hamiltonian[a][b] = ab                        
            """
            if (alpha == beta):
                for i in range(L-1):
                    # Sz is just a function
                    Hamiltonian[count1][count2] += (J*delta*Sz(beta,i)*Sz(beta,i+1))
            b = check(alpha,beta)
            if b:
                del b[0]
                for j in b:
                    Hamiltonian[count1][count2] += (J*0.5*(Sp(beta,j)*Sm(beta,j+1) + Sm(beta,j)*Sp(beta,j+1)))
            count2 += 1
        count1 += 1
    return (np.asmatrix(Hamiltonian))

I mostly just need to know how to make the matrix without having to use as much memory, and then how to put the terms I calculate into the matrix.

Here is my attempt to make the matrix as a sparse matrix.

def SPHamGen(L,N,delta):
"""
Will Generate the hamiltonian matrix, 
Takes parameters:
    L : Number of sites
    N : Number of spin downs
    delta : anistropy 
"""
start = timeit.default_timer()

D = BS.dimension(L,N)
Ham = sp.coo_matrix((D,D))
print Ham
#data = ([0])*D

count1 = 0
Set = BS.getSet(L,N)


data = ([0])*(D*D)
rows = ([0])*(D*D)
cols = ([0])*(D*D)
for alpha in Set:
    count2 = 0
    for beta in Set:
        """
        Compute ab = <alpha|Hamiltonian|beta>
        Then let Hamiltonian[a][b] = ab                        
        """
        if (alpha == beta):
            for i in range(L-1):
                #Hamiltonian[count1][count2] += (J*delta*Sz(beta,i)*Sz(beta,i+1))
                data[count2] += (J*delta*Sz(beta,i)*Sz(beta,i+1)) 
            rows[count2] = count1
            cols[count2] = count2
        b = check(alpha,beta)
        if b:
            del b[0]
            for j in b:
                #Hamiltonian[count1][count2] += (J*0.5*(Sp(beta,j)*Sm(beta,j+1) + Sm(beta,j)*Sp(beta,j+1)))
                data[count2] += (J*0.5*(Sp(beta,j)*Sm(beta,j+1) + Sm(beta,j)*Sp(beta,j+1)))
            rows[count2] = count1
            cols[count2] = count2
        count2 += 1
    count1 += 1
    Ham = Ham + sp.coo_matrix((data,(rows,cols)), shape = (D,D))

time = (timeit.default_timer() - start)
print "\n"+str(time) +"s to calculate H"
#return Ham   
return sparse.csr_matrix(Ham)

Thanks, Phil.

来源：https://stackoverflow.com/questions/45349470/creating-huge-sparse-matrices-in-python