CryptoAPI C++ interop with Java using AES

I am trying to encrypt in C++ using CryptoAPI and decrypt Java using SunJCE. I have gotten the RSA key to work -- and verified on a test string. However, my AES key is not working -- I get javax.crypto.BadPaddingException: Given final block not properly padded.

C++ Encryption:

// init and gen key
HCRYPTPROV provider;
CryptAcquireContext(&provider, NULL, MS_ENH_RSA_AES_PROV, PROV_RSA_AES, CRYPT_VERIFYCONTEXT);

// Use symmetric key encryption
HCRYPTKEY sessionKey;
DWORD exportKeyLen;
CryptGenKey(provider, CALG_AES_128, CRYPT_EXPORTABLE, &sessionKey);

// Export key
BYTE exportKey[1024];
CryptExportKey(sessionKey, NULL, PLAINTEXTKEYBLOB, 0, exportKey, &exportKeyLen);

// skip PLAINTEXTKEYBLOB header
//      { uint8_t bType, uint8_t version, uint16_t reserved, uint32_t aiKey, uint32_t keySize }
DWORD keySize =  *((DWORD*)(exportKey + 8));
BYTE * rawKey = exportKey + 12;

// reverse bytes for java
for (unsigned i=0; i<keySize/2; i++) {
    BYTE temp = rawKey[i];
    rawKey[i] = rawKey[keySize-i-1];
    rawKey[keySize-i-1] = temp;
}

// Encrypt message
BYTE encryptedMessage[1024];
const char * message = "Decryption Works";
BYTE messageLen = (BYTE)strlen(message);
memcpy(encryptedMessage, message, messageLen);
DWORD encryptedMessageLen = messageLen;
CryptEncrypt(sessionKey, NULL, TRUE, 0, encryptedMessage, &encryptedMessageLen, sizeof(encryptedMessage));

// reverse bytes for java
for (unsigned i=0; i<encryptedMessageLen/2; i++) {
    BYTE temp = encryptedMessage[i];
    encryptedMessage[i] = encryptedMessage[encryptedMessageLen - i - 1];
    encryptedMessage[encryptedMessageLen - i - 1] = temp;
}

BYTE byteEncryptedMessageLen = (BYTE)encryptedMessageLen;
FILE * f = fopen("test.aes", "wb");
fwrite(rawKey, 1, keySize, f);
fwrite(&byteEncryptedMessageLen, 1, sizeof(byteEncryptedMessageLen), f);
fwrite(encryptedMessage, 1, encryptedMessageLen, f);
fclose(f);

// destroy session key
CryptDestroyKey(sessionKey);
CryptReleaseContext(provider, 0);

Java Decryption:

try
{
    FileInputStream in = new FileInputStream("test.aes");
    DataInputStream dataIn = new DataInputStream(in);

    // stream key and message
    byte[] rawKey = new byte[16];
    dataIn.read(rawKey);
    byte encryptedMessageLen = dataIn.readByte();
    byte[] encryptedMessage = new byte[encryptedMessageLen];
    dataIn.read(encryptedMessage);

    // use CBC/PKCS5PADDING, with 0 IV -- default for Microsoft Base Cryptographic Provider
    SecretKeySpec sessionKey = new SecretKeySpec(rawKey, "AES");
    Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
    cipher.init(Cipher.DECRYPT_MODE, sessionKey, new IvParameterSpec(new byte[16]));

    cipher.doFinal(encryptedMessage);
}
catch (Exception e) {
  e.printStackTrace();
}

In a similar example I have tried permutations of not reversing the bytes of the key and not reversing bytes in the message. If I encrypt and decrypt with the imported key in java, I get valid results. I can also encrypt and decrypt exclusively in C++.

Questions:

Should I use CBC/PKCS5PADDING? Is this the default for MS_ENH_RSA_AES_PROV?
Is a zeroed IV indeed the default for MS_ENH_RSA_AES_PROV?
Are there any ways to diagnose the specifics of how the key is behaving?
I'd like to stick with standard Java packages instead of installing BouncyCastle, but are there any differences that would make a 3rd party package work better?

I had to do several things to get the message correctly:

Explicitly set KP_MODE to CRYPT_MODE_CBC, and KP_IV to 0
Use NoPadding in Java decryption
Don't reverse the bytes for the key or the message

In terms of diagnosing the problem the most useful piece of advice was to set NoPadding in Java which prevents the BadPaddingException. This allowed me to see the results -- even if wrong.

Strangely, the RSA Java/CryptoAPI interop solution requires the message to be completely byte reversed in order to work with Java, but AES does not expect the key or the message to be byte reversed.

CryptSetKeyParam would not let me use ZERO_PADDING, but when looking at the decrypted bytes, it is clear that CryptoAPI fills with the number of unused bytes. For instance, with a block size of 16, if the last block only uses 9 bytes, then the remaining 5 bytes get the value of 0x05. Does this present a potential security leak? Should I pad all other bytes with random bytes and use only the last byte to signify how much padding is used?

The working code (using the CryptoAPI convention of last byte being pad count) is below (checking of return values from Crypt have been removed for simplicity):

// init and gen key
HCRYPTPROV provider;
CryptAcquireContext(&provider, NULL, MS_ENH_RSA_AES_PROV, PROV_RSA_AES, CRYPT_VERIFYCONTEXT);

// Use symmetric key encryption
HCRYPTKEY sessionKey;
DWORD exportKeyLen;
BYTE iv[32];
memset(iv, 0, sizeof(iv));
DWORD padding = PKCS5_PADDING;
DWORD mode = CRYPT_MODE_CBC;
CryptGenKey(provider, CALG_AES_128, CRYPT_EXPORTABLE, &sessionKey);
CryptSetKeyParam(sessionKey, KP_IV, iv, 0);
CryptSetKeyParam(sessionKey, KP_PADDING, (BYTE*)&padding, 0);
CryptSetKeyParam(sessionKey, KP_MODE, (BYTE*)&mode, 0);

// Export key
BYTE exportKey[1024];
CryptExportKey(sessionKey, NULL, PLAINTEXTKEYBLOB, 0, exportKey, &exportKeyLen);

// skip PLAINTEXTKEYBLOB header
//      { uint8_t bType, uint8_t version, uint16_t reserved, uint32_t aiKey, uint32_t keySize }
DWORD keySize =  *((DWORD*)(exportKey + 8));
BYTE * rawKey = exportKey + 12;

// Encrypt message
BYTE encryptedMessage[1024];
const char * message = "Decryption Works -- using multiple blocks";
BYTE messageLen = (BYTE)strlen(message);
memcpy(encryptedMessage, message, messageLen);
DWORD encryptedMessageLen = messageLen;
CryptEncrypt(sessionKey, NULL, TRUE, 0, encryptedMessage, &encryptedMessageLen, sizeof(encryptedMessage));

BYTE byteEncryptedMessageLen = (BYTE)encryptedMessageLen;
FILE * f = fopen("test.aes", "wb");
fwrite(rawKey, 1, keySize, f);
fwrite(&byteEncryptedMessageLen, 1, sizeof(byteEncryptedMessageLen), f);
fwrite(encryptedMessage, 1, encryptedMessageLen, f);
fclose(f);

// destroy session key
CryptDestroyKey(sessionKey);
CryptReleaseContext(provider, 0);

Java Decryption:

try
{
    FileInputStream in = new FileInputStream("test.aes");
    DataInputStream dataIn = new DataInputStream(in);

    // stream key and message
    byte[] rawKey = new byte[16];
    dataIn.read(rawKey);
    byte encryptedMessageLen = dataIn.readByte();
    byte[] encryptedMessage = new byte[encryptedMessageLen];
    dataIn.read(encryptedMessage);

    // use CBC/NoPadding, with 0 IV -- (each message is creating it's own session key, so zero IV is ok)
    SecretKeySpec sessionKey = new SecretKeySpec(rawKey, "AES");
    Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
    cipher.init(Cipher.DECRYPT_MODE, sessionKey, new IvParameterSpec(new byte[16]));

    byte[] decryptedBlocks = cipher.doFinal(encryptedMessage);

    // check versus expected message
    byte[] expectedBytes = "Decryption Works -- using multiple blocks".getBytes();
    Assert.assertTrue("Incorrect Message" + new String(message), Arrays.equals(message, expectedBytes));
}
catch (Exception e) {
  e.printStackTrace();
}

You are doing too many gyrations for the AES key under Windows. Set it to a known value with CryptImportKey - see for example, WinAES: A C++ AES Class.

You should set CBC mode on Windows using CryptSetKeyParam, KP_MODE, and CRYPT_MODE_CBC. Otherwise, you are using ECB mode (if I recall correctly) Again, see WinAES: A C++ AES Class.

PKCS5 padding is used for symmetric ciphers by default. I don't even recall how to change it (if its possible). I suspect you only other choice is 'no padding'.

Microsoft defaults to a string of 0's for an IV. You will need to set the IV via CryptSetKeyParam and KP_IV.

Q1 & Q2: simply don't rely on defaults. For maintainability, you can choose three options: let everybody find out what the defaults are (not the best option, I think), use comments or simply set all the possible parameters. Personally, I would always opt for the third choice - the other options are too brittle.

Q3 No, if the bits of the key are wrong or in the incorrect order (see below), you will get either a bad padding exception or trash output. What you can do is to use "/NoPadding" in Java during decrypt (or similar in C++). In that way you can see if you have a padding issue by looking at the output. If your plain text is there then you are likely having a padding issue. If only the first block is wrong, you are having trouble with the IV.

Q4 No, not really. Java JCE is working pretty well if you want to stay within Java. Bouncy Castle has (way) more functionality and may have different performance characteristics. You can use other providers to use different key stores (OS dependent or smartcard, for instance), use a performance enhanced (native) implementation etc.

It could be possible that you need the reverse of the key because Java is using big endian and C++ may use little endian. I cannot fanthom that C++ would reverse the bytes of the input/output though. Normally neither of them represent a number, so the order should be identical for both platforms.

Remove the reversal of the bytes, specify all parameters and report back?

来源：https://stackoverflow.com/questions/8763597/cryptoapi-c-interop-with-java-using-aes

标签

java

interop

cryptography

aes

cryptoapi