HSM Error | Private key must be instance of RSAPrivate(Crt)Key or have PKCS#8

Question

Error received while decrypting data when private key is retrieved from HSM.

I have added sunpkcs11 provider in java.security. Hence, NOT adding provider via code. Text gets encrypted successfully. However, while decrypting the encrypted text, I am getting below error at below line:

cipher.init(Cipher.DECRYPT_MODE, privateKey);

What is that i am missing here?

Error:

    Caused by: java.security.InvalidKeyException: Private key must be instance of RSAPrivate(Crt)Key or have PKCS#8 encoding
        at sun.security.pkcs11.P11RSAKeyFactory.implTranslatePrivateKey(P11RSAKeyFactory.java:101) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11KeyFactory.engineTranslateKey(P11KeyFactory.java:132) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11KeyFactory.convertKey(P11KeyFactory.java:65) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11RSACipher.implInit(P11RSACipher.java:199) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11RSACipher.engineInit(P11RSACipher.java:168) [sunpkcs11.jar:1.7.0_85]
        at javax.crypto.Cipher.init(Cipher.java:1068) [jce.jar:1.7.0_85]
        at javax.crypto.Cipher.init(Cipher.java:1012) [jce.jar:1.7.0_85]enter code here

Below is the code:

import java.io.ByteArrayOutputStream;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.cert.Certificate;

import javax.crypto.Cipher;
import javax.xml.bind.DatatypeConverter;

import sun.security.pkcs11.SunPKCS11;

public class App {

    public static void main(String[] args) throws Exception {

        try {
            String passphrase = "mysecretkey";
            SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");
            KeyStore keystore = KeyStore.getInstance("PKCS11", provider);
            keystore.load(null, passphrase.toCharArray());
            String textToEncrypt = "this is my text";
            Certificate cert = keystore.getCertificate("my-SHA1WITHRSA-2048-bits-key");
            PublicKey publicKey = cert.getPublicKey();
            Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", provider);
            cipher.init(Cipher.ENCRYPT_MODE, publicKey);
            String encryptedData = DatatypeConverter.printBase64Binary(cipher.doFinal(textToEncrypt.getBytes()));

            PrivateKey privateKey = (PrivateKey) keystore.getKey("my-SHA1WITHRSA-2048-bits-key",
                    passphrase.toCharArray());
            cipher.init(Cipher.DECRYPT_MODE, privateKey);
            byte[] decodedEncryptedData = DatatypeConverter.parseBase64Binary(encryptedData);
            ByteArrayOutputStream stream = new ByteArrayOutputStream();
            int blocks = decodedEncryptedData.length / 256;
            int offset = 0;
            for (int blockIndex = 0; blockIndex < blocks; blockIndex++) {
                byte[] nextBlock = getNextBlock(decodedEncryptedData, offset);
                stream.write(cipher.doFinal(nextBlock));
                offset += 256;
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

    }

    private static byte[] getNextBlock(byte[] cipherText, int offset) {
        byte[] block = new byte[256];
        System.arraycopy(cipherText, offset, block, 0, 256);
        return block;
    }

}

Answer 1

How I resolved:

Root cause of this issue was that sunpkcs11 provider was getting loaded both statically and dynamically.

i.e. in java.security, provider entry along with cfg path was already added.

Also, in code, provider was initialized again with the cfg file.

This was causing the issue.

After changing:

SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");

TO:

SunPKCS11 sunPKCS11Provider = (SunPKCS11) Security.getProvider("SunPKCS11");

issue got resolved.

Answer 2

I have used following code and issue has been resolved

    SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");
    Security.addProvider(provider);
    KeyStore keystore = KeyStore.getInstance("PKCS11");
    keystore.load(null, passphrase.toCharArray());