scipy sparse matrix: remove the rows whose all elements are zero

Question

I have a sparse matrix which is transformed from sklearn tfidfVectorier. I believe that some rows are all-zero rows. I want to remove them. However, as far as I know, the existi

Answer 1

There aren't existing functions for this, but it's not too bad to write your own:

def remove_zero_rows(M):
  M = scipy.sparse.csr_matrix(M)

First, convert the matrix to CSR (compressed sparse row) format. This is important because CSR matrices store their data as a triple of (data, indices, indptr), where data holds the nonzero values, indices stores column indices, and indptr holds row index information. The docs explain better:

the column indices for row i are stored in indices[indptr[i]:indptr[i+1]] and their corresponding values are stored in data[indptr[i]:indptr[i+1]].

So, to find rows without any nonzero values, we can just look at successive values of M.indptr. Continuing our function from above:

  num_nonzeros = np.diff(M.indptr)
  return M[num_nonzeros != 0]

The second benefit of CSR format here is that it's relatively cheap to slice rows, which simplifies the creation of the resulting matrix.