Stopping decryption before EOF throws exception: Padding is invalid and cannot be removed

Question

This is the scenario we have: We have huge encrypted files, in the order of gigabytes that we can decrypt correctly if we read them until the end. The problem arises when we ar

Answer 1

This exception is thrown during Dispose(true). Throwing from Dispose is already a design flaw (https://docs.microsoft.com/en-us/visualstudio/code-quality/ca1065-do-not-raise-exceptions-in-unexpected-locations#dispose-methods), but it's even worse since this exception is thrown even before the underlying stream is closed.

This means that anything that receives a Stream that might be a CryptoStream needs to work around this and either close the underlying Stream themselves in a 'catch' block (essentially needing a reference to something completely unrelated), or somehow warn all listeners that the stream may still be open (e.g., "don't try to delete the underlying file -- it's still open!").

No, in my book, this is a pretty big oversight, and the other answers don't seem to address the fundamental issue. CryptoStream takes ownership of the passed-in stream, so it takes on the responsibility to close the underlying stream before control leaves Dispose(true), end of story.

Ideally, it should also never throw under circumstances that are not truly exceptional (such as "we stopped reading early, because the decrypted data is in the wrong format and it's a waste of time to continue reading").

Our solution was basically this (update: but be warned -- as Will Krause pointed out in the comments, this could leave sensitive information lying around in the private _InputBuffer and _OutputBuffer fields that can be accessed via reflection. Versions 4.5 and above of the .NET Framework don't have this problem.):

internal sealed class SilentCryptoStream : CryptoStream
{
    private readonly Stream underlyingStream;

    public SilentCryptoStream(Stream stream, ICryptoTransform transform, CryptoStreamMode mode)
        : base(stream, transform, mode)
    {
        // stream is already implicitly validated non-null in the base constructor.
        this.underlyingStream = stream;
    }

    protected override void Dispose(bool disposing)
    {
        try
        {
            base.Dispose(disposing);
        }
        catch (CryptographicException)
        {
            if (disposing)
            {
                this.underlyingStream.Dispose();
            }
        }
    }
}

Answer 2

Close calls Dispose(true) which calls FlushFinalBlock which throws the exception, because this is not really the final block.

You can prevent this by overriding the Close method so that it doesn't call FlushFinalBlock:

public class SilentCryptoStream : CryptoStream {
    public SilentCryptoStream(Stream stream, ICryptoTransform transform, CryptoStreamMode mode) :
        base(stream, transform, mode) {
    }

    public override void Close() {
        this.Dispose(false);
        GC.SuppressFinalize(this);
    }
}

(You also need to manually close the underlying stream.)

is it valid to check if reader.EndOfStream is false and then capture the CryptographicException

I think it's OK.

Answer 3

My solution was to, in my derived class, add this to my Dispose(bool) override:

    protected override void Dispose(bool disposing)
    {
        // CryptoStream.Dispose(bool) has a bug in read mode. If the reader doesn't read all the way to the end of the stream, it throws an exception while trying to
        // read the final block during Dispose(). We'll work around this here by moving to the end of the stream for them. This avoids the thrown exception and
        // allows everything to be cleaned up (disposed, wiped from memory, etc.) properly.
        if ((disposing) &&
            (CanRead) &&
            (m_TransformMode == CryptoStreamMode.Read))
        {
            const int BUFFER_SIZE = 32768;
            byte[] buffer = new byte[BUFFER_SIZE];

            while (Read(buffer, 0, BUFFER_SIZE) == BUFFER_SIZE)
            {
            }
        }

        base.Dispose(disposing);
        ...

By making sure the stream is always read to the end, the internal issue in the CryptStream.Dispose is avoided. Of course, you need to weigh this against the nature of what you are reading, to be sure it doesn't have a negative impact. Only use it against a source of a known finite length.

Answer 4

Can you turn off padding?

// aes.Padding = PaddingMode.PKCS7;
aes.Padding = PaddingMode.None;

Answer 5

As I understand it, the exception is thrown when the last byte read is not a valid padding byte. When you intentionally close the stream early, the last byte read will most likely be considered "invalid padding" and the exception is thrown. Since you're ending intentionally, you should be safe ignoring the exception.