JWT ES256 base64 encode EC key

Question

When I try to base64urlencode my public key x and y (ES256) for use with JSON Web Token, I run into padding problems.

I have