This is a chicken-egg problem.
You will then need to to find where to store the password for the keystore and and so on.....
Depending on your security requirements and needs, you could enrypt the data using symmetric encryption and retrieve the password via a remote server (your server) using client side certificate authentication.
Or you could encrypt the data using symmetric enryption and have the password hardcoded in your jar, which is not safe but requires some effort to find it and you could delegate responsibility to the file system permissions on who can access your files.