file ~/.serverauth.123 doesn't exist error when trying to run startx without sudo on EC2 instance

Question

I am using a Amazon EC2 g4 instance with AL2 OS.

I was earlier using gdm to start X but due to some issues we are not using gdm and instead trying to start X directly