Whay ifft2 is not working but fft2 is fine?

Question

I want to implement ifft2 using DFT matrix. The following code works for fft2.

import numpy as np

def DFT_matrix(N):
    i, j = np.meshgrid(np.arange(N), np.