Java TLS 1.2 server : AES-GCM decryption

问题

I'm currently working on a Java TLS server. (I posted here a few days ago regarding KeyExchange signature)

I'm now trying to decrypt TLS message encoded using AES-GCM. The server already handles CBC but since it's vulnerable to POODLE we'd like to do GCM instead. I'll try to explain as best as I can :)

For this code we are using Java 8u91, Netty 3.9.0. We do not use BouncyCastle and we don't intend to, we would like to stick with the JDK.

The code !

/**
 * Deciphers the fragment and returns the deciphered version of it
 * 
 * @param fragment
 *            to decrypt
 * @return the decrypted fragment
 * @throws InvalidKeyException
 * @throws InvalidAlgorithmParameterException
 * @throws IllegalBlockSizeException
 * @throws BadPaddingException
 */
    private ChannelBuffer decodeCiphered(ChannelBuffer fragment, short contentType, byte[] version) throws InvalidKeyException,
    InvalidAlgorithmParameterException, IllegalBlockSizeException, BadPaddingException {
    if (readCipher != null) {
        byte[] ciphered = fragment.array();
        if (session.cipherSuite.cipher.type == CipherType.stream) {
            /* ... */
        } else if (session.cipherSuite.cipher.type == CipherType.block) {
            /* ... */
        } else if (session.cipherSuite.cipher.type == CipherType.aead) {
            byte[] nonce = concat(clientWriteIV.getIV(), fragment.copy(0, 8).array());
            // ClientWriteIV + explicit nonce (first 8 bytes of encrypted data)

            readCipher.init(Cipher.DECRYPT_MODE, clientWriteKey,
                new GCMParameterSpec(session.cipherSuite.cipher.block * 8, nonce));

            ChannelBuffer aad = ChannelBuffers.buffer(13);
            //seq_num(8B) + TLSCompressed.type(1B) + TLSCompressed.version(2B) + TLSCompressed.length(2B)

            aad.writeLong(writeSequence.longValue());
            aad.writeByte(contentType);
            aad.writeBytes(version);
            aad.writeShort(fragment.readableBytes() - 8 - 16);
            readCipher.updateAAD(aad.array());
            ciphered = readCipher.doFinal(ciphered, 8, ciphered.length - 8);
            fragment = ChannelBuffers.wrappedBuffer(ciphered);
        }
    return fragment;
}

/**
 * Generates the different needed keys.
 */
private void generateKeys() {
    byte[] keyBlock = null;
    if (session.cipherSuite.cipher.type == CipherType.aead) {
        keyBlock = new byte[2 * session.cipherSuite.cipher.key
                    + 8];

        try {
            prf(keyBlock, session.masterSecret, KEY_EXPANSION_LABEL, concat(serverRandom, clientRandom));
        } catch (GeneralSecurityException e) {
        }
        clientWriteKey = new SecretKeySpec(keyBlock,0 , session.cipherSuite.cipher.key, session.cipherSuite.cipher.kalgo);
        serverWriteKey = new SecretKeySpec(keyBlock, session.cipherSuite.cipher.key, session.cipherSuite.cipher.key, session.cipherSuite.cipher.kalgo);
        clientWriteIV = new IvParameterSpec(keyBlock, 2 * session.cipherSuite.cipher.key, 4);
        serverWriteIV = new IvParameterSpec(keyBlock, 2 * session.cipherSuite.cipher.key + 4, 4);

    } else {
        /* ... 
            For CBC
        */
    }
}

I am constantly getting the following error :

javax.crypto.AEADBadTagException: Tag mismatch!
at com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:524)
at com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1023)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:960)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:824)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:436)
at javax.crypto.Cipher.doFinal(Cipher.java:2223)
at com.seemy.codec.tls.TlsCodec.decodeCiphered(TlsCodec.java:525)

It seems usually this means the AAD isn't right. But as far as I can tell from the RFC 5246 Section 6.2.3.3 the AAD needs to be as follows :

The additional authenticated data, which we denote as additional_data, is defined as follows:

  additional_data = seq_num + TLSCompressed.type +
                    TLSCompressed.version + TLSCompressed.length;
where "+" denotes concatenation.

I'm at lost to find what I'm doing wrong to decrypt this message. The "Tag Mismatch" error isn't of much help to find what's wrong, so I hope someone here can help me :)

Thank you and have a nice day !

Edit 1 : I was putting a short instead of a byte in contentVersion, however the result is still the same. I get only Tag Mismatch errors...

Edit 2 : As suggested by @dave_thompson_085 doFinal is now called on (ciphered, 8, ciphered.length-8) instead of (ciphered, 0, ciphered.length) to exclude the explicit_nonce.

I also checked the aad and nonce values with more scrutiny.

The explicit_nonce part of the nonce corresponds to the packet I'm seeing in Wireshark. I am now worried the client_write_IV isn't properly generated :|

As for the aad I discovered something a bit weird : the length is 0.

The message I'm getting from the client is 40 Bytes long, minus 8 for the explicit_nonce part, minus the MAC length which is 32 for SHA-256(I'm trying to get tls_ecdhe_rsa_with_aes_128_gcm_sha256 working).

Consequently if I try tls_ecdhe_rsa_with_aes_256_gcm_sha384, the length ends up being -16. This seems wrong to me.

Edit 3 : Again I did as @dave_thompson_085 suggested and modified my code to have the length as follow : aad.writeShort(fragment.readableBytes() - 8 - 16); whereas before I did aad.writeShort(fragment.readableBytes()-8-session.cipherSuite.mac.len);

It seems to work well when talking with chrome (even though it fails after when I send my own encrypted message but this will require another post later if I can't fix it by myself) but when trying with open_ssl client I get a Tag Mismatch again. Firefox & Safari just send me protocol_version alerts…

来源：https://stackoverflow.com/questions/38436853/java-tls-1-2-server-aes-gcm-decryption

标签

java

aes

tls1.2

aes-gcm