Is my encryption safe as openPGP/SMIME?

Question

I\'m trying to write a simple file enc/decryption within a larger project. I\'d like to avoid libgpgme because of license issues. The openPGP standard is to complex for the

Answer 1

Since this is a new format, you should use OAEP padding. Just change RSA_PKCS1_PADDING to RSA_PKCS1_OAEP_PADDING. You actually don't need to encrypt the IV (it can't hurt as far as I can tell, and it might help).

Otherwise, this method should be fine so long as RSA_size(rsa)==16. Of course, the private key must not be knowable by anyone who should not be able to decrypt the file.

Answer 2

Encryption is a topic where things are easy to make "work" - but hard to make secure. When in doubt (and doubly so when not in doubt), pick a widely recognized standard and implement precisely to spec. The idea of encrypting the key with a public-private algorithm, then packing the IV in as well is sound in theory, but I'm not sure what the implications of encrypting the IV as well are, and what happens if the attacker starts flipping bits in the encrypted data? Etc. It looks sound, but again, I would strongly recommend simply implementing a published spec precisely.

I would recommend just implementing S/MIME, using a binary transfer encoding. S/MIME is recognized as being a secure specification, there are libraries implementing all the hard parts, and most importantly, you can test your implementation against other implementations to make sure you're not out of spec.

Answer 3

Since you are using the OpenSSL envelope-encryption functions anyway, you should just directly use the EVP_SealInit() / EVP_SealUpdate() / EVP_SealFinal() functions. These functions take care of generating the symmetric key and IV, encrypting the data with the symmetric key and encrypting the symmetric key with the recipient(s) RSA key(s).

Once thing that you are not taking care of is authenticity. Under CBC mode it is possible for an attacker to make certain predictable changes to the plaintext, even if they can't read it. To detect this, you should either calculate a HMAC over the encrypted message (using a seperate symmetric key to that used for encryption), or sign the encrypted message (eg. with EVP_SignInit() / EVP_SignUpdate() / EVP_SignFinal()).

Answer 4

Some observations:

• The EVP_BytesToKey function is meant to create a key and initialization vector from a password and salt, not from random data. It will work, but you could also simply use the random bytes directly as key and initialization vector. (Make sure you are using a secure PRNG, I'm not sure what RAND_bytes actually does.)

• The initialization vector does not need to be secret, CBC mode should be secure with a non-encrypted IV. (This does not hurt, though.)

• The RSA encryption looks good (but you might want to use another padding, as David said).

• As Serdalis said, you should also protect your file against modifications. Any keyed MAC will do (most common are HMAC build on a key and a hash function). Apply the MAC after encryption.