TripleDES in CFB mode, C# and Crypto++ differs

Question

Here is my problem: I've got a legacy code in C++ (using crypto++ v5.6.1) and I develop a new one in C# (.NET 3.5 using System.Security.Cryptography). I can't change the C++ code, but I need to be able to decrypt data previously encrypted and previous applications has to be able to decrypt the data I will crypt with my new C# code.

The algorithm used is TripleDES with CFB cipher mode in both cases, but in the end, the encrypted data are not the same, the number of bytes are identical as well as the first byte, but apart from that all other bytes are different.

The padding is manually done (adding zeros) in the C++ code. So I set the PaddingValue to PaddingMode.Zeros. (I also tried to add zeros in the end manually in the C# code, it didn't change anything).

I tried using different System.Text.Encoding but the result is the same (in fact the tested characters are "pure" ASCII (i.e.: between 0 & 126)).

The value of the MandatoryBlockSize(), in the C++ code, is 8, so I set the FeedbackSize to 8 too. But if I understand it write, it is in fact the size of my IV, isn't it ?

The key size is 24 bytes (3 different keys) and the IV is 8 bytes long. They are both the same in the 2 codes.

If I use CBC mode instead in both cases the results are the same (but, as I said, I can't change the legacy code ...), OFB & CTS mode throw exceptions (unavailable for one and incompatible for the other) on my .NET application, so I can't compare results.

I tried using Mono, with .Net versions 3.5 & 4.0, or using visual, with .Net 3.5 or 4.0, and the 4 encrypted results are the same but it differs from the original result.

Now I really don't know what to test ... I'd rather not wrap Crypto++ in a C++/CLI project to use it instead of the System.Security.Cryptography.

Does someone have an advice or can tell what am I doing wrong ?

Here is the C++ code:

void *CryptData(BYTE *bDataIn, LONG lIn, LONG *lOut, byte* key, byte* iv)
{
    byte    *bIn;
    byte    *bOut;
    LONG    l2,lb;

    CFB_FIPS_Mode<DES_EDE3>::Encryption encryption_DES_EDE3_CFB;
    encryption_DES_EDE3_CFB.SetKeyWithIV(key, sizeof(key), iv, sizeof(iv));

    lb = encryption_DES_EDE3_CFB.MandatoryBlockSize();
    l2 = ((lIn + lb - 1)/lb)*lb;

    bIn = (byte*)malloc(l2);
    bOut = (byte*)malloc(l2);

    memset(bIn,0,l2);
    memset(bOut,0,l2);
    memcpy(bIn,bDataIn,lIn);

    encryption_DES_EDE3_CFB.ProcessString(bOut, bIn, l2);

    *lOut = l2;

    return bOut;
}

Here is the C# code:

public FibxCrypt()
{
    _cryptoAlgo = new TripleDESCryptoServiceProvider();
    //_cryptoAlgo.GenerateKey();
    _cryptoAlgo.Key = _key;
    //_cryptoAlgo.GenerateIV();
    _cryptoAlgo.IV = _iv;
    _cryptoAlgo.Mode = CipherMode.CFB;
    _cryptoAlgo.Padding = PaddingMode.Zeros;

    _encoding = new UTF8Encoding();
}

private MemoryStream EncryptingString(string plainText, out long encryptSize)
{
    // Check arguments. 
    if (plainText == null || plainText.Length <= 0)
        throw new ArgumentNullException("plainText");

    // Create a decrytor to perform the stream transform.
    ICryptoTransform encryptor = _cryptoAlgo.CreateEncryptor();

    // Create the streams used for encryption. 
    //using (MemoryStream msEncrypt = new MemoryStream())
    MemoryStream msEncrypt = new MemoryStream();

    encryptSize = ((plainText.Length + _cryptoAlgo.FeedbackSize - 1) / _cryptoAlgo.FeedbackSize) * _cryptoAlgo.FeedbackSize;

    using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
    {
        using (StreamWriter swEncrypt = new StreamWriter(csEncrypt, _encoding))
        {
            //Write all data to the stream.
            swEncrypt.Write(plainText);
        }
    }

    // Return the encrypted memory stream. 
    return msEncrypt;
}

EDIT : I tried to use the Encryptor directly, instead of using the streams and I've got the same problem.

private MemoryStream EncryptingString(string plainText, out long encryptSize)
{
    // Check arguments. 
    if (plainText == null || plainText.Length <= 0)
        throw new ArgumentNullException("plainText");

    ICryptoTransform encryptor = _cryptoAlgo.CreateEncryptor();

    byte[] cipherData = encryptor.TransformFinalBlock(
        _encoding.GetBytes(plainText), 0, plainText.Length);

    // Return the encrypted memory stream. 
    return msEncrypt;
}

Answer 1

The FeedBackSize you have changed, relates to the CFB mode of operation (msdn documentation). Therefore you should also check that Feedback size in C++ and C# are the same.

I believe that your bug could be maligned BlockSizes between the C++ code and the C# code. Have you tried setting BlockSize = 8 in the C# implementation?

Answer 2

These are not correct:

CFB_FIPS_Mode<DES_EDE3>::Encryption enc;
enc.SetKeyWithIV(key, sizeof(key), iv, sizeof(iv));

sizeof(key) and sizeof(iv) returns the size of the pointers, not the size of the security parameters. You should use this instead:

enc.SetKeyWithIV(key, DES_EDE3::DEFAULT_KEYLENGTH, iv, DES_EDE3::BLOCKSIZE);

---

If it works for .Net, then you should prefer to increase the feedback size for libraries like Mcrypt and .Net; and not reduce the feedback size in Crypto++. That's because some modes lose security when the feedback size is not the full block size.

I don't know if this will work with .Net, but its something you should consider or try:

public FibxCrypt()
{
    _cryptoAlgo = new TripleDESCryptoServiceProvider();
    _cryptoAlgo.Key = _key;
    _cryptoAlgo.IV = _iv;
    _cryptoAlgo.Mode = CipherMode.CFB;
    _cryptoAlgo.Padding = PaddingMode.Zeros;

    // Add this:
   _cryptoAlgo.FeedbackSize = _cryptoAlgo.BlockSize;
}

If you can't adjust the feedback size in .Net, then here's how to change feedback size in Crypto++. You setup a AlgorithmParameters to hold the feedback size parameter, and then you call SetKey with the additional parameters:

void *CryptData(BYTE *bDataIn, LONG lIn, LONG *lOut, byte* key, byte* iv)
{
    AlgorithmParameters params = MakeParameters(Name::FeedbackSize(), 1 /*8-bits*/)
                                               (Name::IV(), ConstByteArrayParameter(iv, DES_EDE3::BLOCKSIZE));
    CFB_FIPS_Mode<DES_EDE3>::Encryption enc;
    enc.SetKey(key, 24, DES_EDE3::DEFAULT_KEYLENGTH);

    ...
}

Its not clear to me if CFB mode operating in FIPS mode allows such a small feedback size. If it throws an exception, then you will need to use just CFB_Mode.

AlgorithmParameters may look a little odd because of the operator() overload. You can read about it at NameValuePairs on the Crypto++ wiki. Other wiki pages of interest are TripleDES and CFB Mode.

----

Another thing to watch out for is text encoding. It usually causes interoperability issues in .Net and Java due to UTF-16. UTF-8 and ASCII cause the least amount of problems. You should be OK since you encoding = new UTF8Encoding().

But if things still don't work for you, then you a byte message that is not encoded or interpreted. For example, use this in both .Net and Crypto++:

byte msg[4] = { 0x01, 0x02, 0x03, 0x04 };

The four bytes are not interpreted, so it side steps encoding issues.