Unable to decrypt data encoded via OpenSSL on iPhone

Question

Kind of a followup to my previous question to: How do I get the initialization vector (iv) from OpenSSL encrypted data

I\'m using OpenSSLcommand line ut

Answer 1

While the iv is handles incorrectly that is the least of the problems.

A decode error sounds like incorrect argument lengths since any random iv, key and data should be valid input. (my wife agrees and she does this stuff professionally.) Check things like the key and data length after converting them to NSData. Note that passing encrypted data with an incorrect or incompatible padding will also result in a decoding error.

Write a test for Base64, your iOS code vs openssl.

Work up to the solution from simpler tests.

For example drop the base64 until you get the encryption top work. Try simple data, say one block length of 0's, padding can be a problem. Try a simpler key such as all 0's. You can use OPENSSL on the Mac Terminal command line.

Once the basic encryption is working add back in the needed functionality.

For openssl from the command line use input and output files, they will handle binary so you will not have that hurdle at least initially. Here is a sample:

(file_orig.txt contains: "1234567890123456")

openssl enc -e -aes-128-cbc -K 00ff349830193845af43984758690213 -p -iv 0 -nosalt -in file_orig.txt -out file_aes.txt

which prints out the key it generated as well as the iv it used:

key=00ff349830193845af43984758690213
iv =00000000000000000000000000000000

Then you can read the same data files in your iOS method.

Here is an iOS method that uses the files openssl creates:
(put the key openssl output into the file key-hex-openssl.txt)

NSData *keyHexData = [@"00ff349830193845af43984758690213" dataUsingEncoding:NSUTF8StringEncoding];
NSData *testData   = [NSData dataWithContentsOfFile:@"yourDirectoryPath/file_aes.txt"];
NSData *clearData  = [NSData dataWithContentsOfFile:@"yourDirectoryPath/file_orig.txt"];

NSLog(@"keyHexData: %@", keyHexData);
NSLog(@"testData:   %@", testData);
NSLog(@"clearData:  %@", clearData);

unsigned char keyBytes[16];
unsigned char *hex = (uint8_t *)keyHexData.bytes;

char byte_chars[3] = {'\0','\0','\0'};
for (int i=0; i<16; i++) {
    byte_chars[0] = hex[i*2];
    byte_chars[1] = hex[(i*2)+1];
    keyBytes[i] = strtol(byte_chars, NULL, 16);
}
NSData *keyData = [NSData dataWithBytes:keyBytes length:16];
NSLog(@"keyData:    %@", keyData);

NSData *ivData = [NSData dataWithBytes:(char []){0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0} length:16];
NSLog(@"ivData:     %@", ivData);

CCCryptorStatus ccStatus   = kCCSuccess;
size_t          cryptBytes = 0;    // Number of bytes moved to buffer.
NSMutableData  *clearOut   = [NSMutableData dataWithLength:testData.length];

ccStatus = CCCrypt(kCCDecrypt,
                   kCCAlgorithmAES128,
                   kCCOptionPKCS7Padding,
                   keyData.bytes, 
                   kCCKeySizeAES128,
                   ivData.bytes,
                   testData.bytes,
                   testData.length,
                   clearOut.mutableBytes,
                   clearOut.length,
                   &cryptBytes);

if (ccStatus != kCCSuccess) {
    NSLog(@"CCCrypt status: %d", ccStatus);
}

clearOut.length = cryptBytes;
NSLog(@"clearOut:   %@", clearOut);
keyHexData: <41393641 34344436 31343245 43463546 33444339 30303038 46453941 34383838>
testData:   <86a8b306 0f33db02 01e77e66 af5bcb3a>
clearData:  <31323334 35363738 39303132 33343536>
keyData:    <a96a44d6 142ecf5f 3dc90008 fe9a4888>
ivData:     <00000000 00000000 00000000 00000000>
clearOut:   <31323334 35363738 39303132 33343536>

Note that clearData has been recovered into clearOut

This demonstrates encrypting with openssl and decrypting with CommonCrypto.

Problems to be overcome:
1) Base64 needs to be added

This is a starting point to complete the encryption needed.

Answer 2

To me your 00ff349830193845af43984758690213 looks a lot more like hex digits than Base64. The fact that there are 16 bytes worth of hex makes it a good bet that this is the IV expressed in hex, not in Base64.

Answer 3

You're providing CCCrypt the UTF8 representation of your key and IV when the function wants raw bytes. If you are storing your key and IV in string form, where the string is the hexadecimal representation of the bytes, then you need to convert this string of hex digits back to raw bytes.

For example, let's take your IV of all zero bytes.

You provided OpenSSL with the IV "00000000000000000000000000000000". OpenSSL takes that string and converts each two hex digits into their respective byte, and comes up with these 16 bytes:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (This is an array of 16 zero bytes)

You, however, take the IV string and get the UTF8 representation of it.

// fyi, I plan to replace this later with a random iv
NSData *ivData = [@"00000000000000000000000000000000" dataUsingEncoding:NSUTF8StringEncoding];

What you end up with are these bytes:

30 30 30 30 30 30 30 30 ...

Because the character '0' in UTF8 is 0x30. So convert your hex representations into bytes, not UTF8 characters, and your key and IV will match OpenSSL's.

OpenSSL has a function called set_hex that does the conversion from string to bytes (which in C are held in an unsigned char array).

#define BIO_printf fprintf
#define bio_err stderr
int set_hex(char *in, unsigned char *out, int size)
        {
        int i,n;
        unsigned char j;

        n=strlen(in);
        if (n > (size*2))
                {
                BIO_printf(bio_err,"hex string is too long\n");
                return(0);
                }
        memset(out,0,size);
        for (i=0; i<n; i++)
                {
                j=(unsigned char)*in;
                *(in++)='\0';
                if (j == 0) break;
                if ((j >= '0') && (j <= '9'))
                        j-='0';
                else if ((j >= 'A') && (j <= 'F'))
                        j=j-'A'+10;
                else if ((j >= 'a') && (j <= 'f'))
                        j=j-'a'+10;
                else
                        {
                        BIO_printf(bio_err,"non-hex digit\n");
                        return(0);
                        }
                if (i&1)
                        out[i/2]|=j;
                else
                        out[i/2]=(j<<4);
                }
        return(1);
        }

E.g.,

    char iv_str[] = "12345678901234567890123456789012";
    unsigned char iv[16];
    if( !set_hex(iv_str, iv, sizeof(iv)) )
    {
        // Handle error where string was not a well-formed IV
    }
    printf("IV: "); for(int i=0;i<sizeof(iv); ++i) { printf("%02x", iv[i]); } printf("\n");