It finally worked with the go built in x509.CreateCertificate,
the problem was that I did not set the IsCA:true flag,
I only set the x509.KeyUsageCertSign which made creating the self signed certificate work, but crashed while verifying the cert chain.