Tensorflow Sparse Matrix calculation takes more time than Dense Matrix

Question

I am trying to optimize my model by moving from dense matrix calculations to sparse matrices on Tensorflow. But unexpectedly sparse matrix calculations take significantly more t