Client Certificate authentication without local truststore

Question

OK, this might sound strange at first, so please bear with me :-)

The problem I need to solve is this:
I need to enable client authentication in a Spring Boot applic

Answer 1

What I want to achieve basically boils down to one issue:
Instead of loading the truststore from a file, the truststore has to be created in-memory, based on data from the secure configuration storage.
This turned out to be a bit tricky, but absolutely possible.

Creating a truststore is easy:

KeyStore ts = KeyStore.getInstance(KeyStore.getDefaultType());
ts.load(null);

for (Certificate cert : certList) {
    ts.setCertificateEntry(UUID.randomUUID().toString(), cert);
}

However, supplying it to the SSL processing pipeline is a bit tricky. Basically, what we need to do is to provide an implementation of X509ExtendedTrustManager that uses the truststore that we created above.
To make this implementation known to the SSL processing pipeline, we need to implement our own provider:

public class ReloadableTrustManagerProvider extends Provider {
    public ReloadableTrustManagerProvider() {
        super("ReloadableTrustManager", 1, "Provider to load client certificates from memory");
        put("TrustManagerFactory." + TrustManagerFactory.getDefaultAlgorithm(), ReloadableTrustManagerFactory.class.getName());
    }
}

This provider in turn uses a TrustManagerFactorySpi implementation:

public class ReloadableTrustManagerFactory extends TrustManagerFactorySpi {

    private final TrustManagerFactory originalTrustManagerFactory;

    public ReloadableTrustManagerFactory() throws NoSuchAlgorithmException {
        ProviderList originalProviders = ProviderList.newList(
                Arrays.stream(Security.getProviders()).filter(p -> p.getClass() != ReloadableTrustManagerProvider.class)
                        .toArray(Provider[]::new));

        Provider.Service service = originalProviders.getService("TrustManagerFactory", TrustManagerFactory.getDefaultAlgorithm());
        originalTrustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm(), service.getProvider());
    }

    @Override
    protected void engineInit(KeyStore keyStore) throws KeyStoreException {
    }

    @Override
    protected void engineInit(ManagerFactoryParameters managerFactoryParameters) throws InvalidAlgorithmParameterException {
    }

    @Override
    protected TrustManager[] engineGetTrustManagers() {
        try {
            return new TrustManager[]{new ReloadableX509TrustManager(originalTrustManagerFactory)};
        } catch (Exception e) {
            return new TrustManager[0];
        }
    }
}

More about originalTrustManagerFactory and ReloadableX509TrustManager later.
Finally, we need to register the provider in a way that makes it the default one, so that the SSL pipeline will use it:

Security.insertProviderAt(new ReloadableTrustManagerProvider(), 1);

This code can be execute in main, before SpringApplication.run.

To recap: We need to insert our provider into the list of security providers. Our provider uses our own trust manager factory to create instances of our own trust manager.

Two things are still missing:

1. The implementation of our trust manager
2. The explanation for originalTrustManagerFactory

First, the implementation (based on https://donneyfan.com/blog/dynamic-java-truststore-for-a-jax-ws-client):

public class ReloadableX509TrustManager extends X509ExtendedTrustManager implements X509TrustManager {
    private final TrustManagerFactory originalTrustManagerFactory;
    private X509ExtendedTrustManager clientCertsTrustManager;
    private X509ExtendedTrustManager serverCertsTrustManager;
    private ArrayList<Certificate> certList;
    private static Log logger = LogFactory.getLog(ReloadableX509TrustManager.class);

    public ReloadableX509TrustManager(TrustManagerFactory originalTrustManagerFactory) throws Exception {
        try {
            this.originalTrustManagerFactory = originalTrustManagerFactory;
            certList = new ArrayList<>();
            /* Example on how to load and add a certificate. Instead of loading it here, it should be loaded externally and added via addCertificates
            // Should get from secure configuration store
            String cert64 = "base64 encoded certificate";
            byte encodedCert[] = Base64.getDecoder().decode(cert64);
            CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
            X509Certificate cert = (X509Certificate) certFactory.generateCertificate(new ByteArrayInputStream(encodedCert));
            certList.add(cert); */
            reloadTrustManager();
        } catch (Exception e) {
            logger.fatal(e);
            throw e;
        }
    }

    /**
     * Removes a certificate from the pending list. Automatically reloads the TrustManager
     *
     * @param cert is not null and was already added
     * @throws Exception if cannot be reloaded
     */
    public void removeCertificate(Certificate cert) throws Exception {
        certList.remove(cert);
        reloadTrustManager();
    }

    /**
     * Adds a list of certificates to the manager. Automatically reloads the TrustManager
     *
     * @param certs is not null
     * @throws Exception if cannot be reloaded
     */
    public void addCertificates(List<Certificate> certs) throws Exception {
        certList.addAll(certs);
        reloadTrustManager();
    }

    @Override
    public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
        clientCertsTrustManager.checkClientTrusted(chain, authType);
    }

    @Override
    public void checkClientTrusted(X509Certificate[] x509Certificates, String s, Socket socket) throws CertificateException {
        clientCertsTrustManager.checkClientTrusted(x509Certificates, s, socket);
    }

    @Override
    public void checkClientTrusted(X509Certificate[] x509Certificates, String s, SSLEngine sslEngine) throws CertificateException {
        clientCertsTrustManager.checkClientTrusted(x509Certificates, s, sslEngine);
    }

    @Override
    public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
        serverCertsTrustManager.checkServerTrusted(chain, authType);
    }

    @Override
    public void checkServerTrusted(X509Certificate[] x509Certificates, String s, Socket socket) throws CertificateException {
        serverCertsTrustManager.checkServerTrusted(x509Certificates, s, socket);
    }

    @Override
    public void checkServerTrusted(X509Certificate[] x509Certificates, String s, SSLEngine sslEngine) throws CertificateException {
        serverCertsTrustManager.checkServerTrusted(x509Certificates, s, sslEngine);
    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return ArrayUtils.addAll(serverCertsTrustManager.getAcceptedIssuers(), clientCertsTrustManager.getAcceptedIssuers());
    }

    private void reloadTrustManager() throws Exception {
        KeyStore ts = KeyStore.getInstance(KeyStore.getDefaultType());
        ts.load(null);

        for (Certificate cert : certList) {
            ts.setCertificateEntry(UUID.randomUUID().toString(), cert);
        }

        clientCertsTrustManager = getTrustManager(ts);
        serverCertsTrustManager = getTrustManager(null);
    }

    private X509ExtendedTrustManager getTrustManager(KeyStore ts) throws NoSuchAlgorithmException, KeyStoreException {
        originalTrustManagerFactory.init(ts);
        TrustManager tms[] = originalTrustManagerFactory.getTrustManagers();
        for (int i = 0; i < tms.length; i++) {
            if (tms[i] instanceof X509ExtendedTrustManager) {
                return (X509ExtendedTrustManager) tms[i];
            }
        }

        throw new NoSuchAlgorithmException("No X509TrustManager in TrustManagerFactory");
    }
}

This implementation has a few notable points:

1. It actually delegates all work to the normal default trust manager. To be able to obtain it, we need to have the default trust manager factory that would normally be used by the SSL pipeline. That's why we pass it as the parameter originalTrustManagerFactory in the constructor.
2. We are actually using two different trust manager instances: One for validating client certificates - which is used when a client sends a request to us and authenticates itself with a client certificate - and another one for validating server certificates - which is used when we send a request to a server using HTTPS. For validating the client certificates, we create a trust manager with our own truststore. This will contain only the certificates that are stored in our secure configuration store and therefore it will not contain any of the root CAs that Java usually trusts. If we would use this trust manager for requests to a HTTPS URL where we are the client, the request would fail as we would be unable to verify the validity of the server's certificate. Therefore, the trust manager for the server certificate validation is created without passing a truststore and will therefore use the default Java truststore.
3. getAcceptedIssuers needs to return the accepted issuers from both our trust managers, because in this method we don't know if the certificate validation is happening for a client or a server certificate. This has the small drawback that our trustmanager would also trust servers which are using our self signed client certificates for their HTTPS.

To make all of this work, we need to enable ssl client authentication:

server.ssl.key-store: classpath:keyStore.p12 # secures our API with SSL. Needed, to enable client certificates handling
server.ssl.key-store-password: very-secure
server.ssl.client-auth: need

Because we are creating our own truststore, we don't need the setting server.ssl.trust-store and its related settings