How to deal with huge memory

Question

I have a piece of code as follows (it is a recursion relation), and it takes much much memory. The matrix size gets large when we increase pN as one can see in the