Can the Encoding API decode a Stream/noncontinuous bytes?

Question

Usually we can get a string from a byte[] using something like

var result = Encoding.UTF8.GetString(bytes);

However,

Answer 1

The Encoding class can't deal with that directly, but the Decoder returned from Encoding.GetDecoder() can (indeed, that's its entire reason for existing). StreamReader uses a Decoder internally.

It's slightly fiddly to work with though, as it needs to populate a char[], rather than returning a string (Encoding.GetString() and StreamReader normally handle the business of populating the char[]).

The problem with using a MemoryStream is that you're copying all of the bytes from one array to another, for no gain. If all of your buffers are the same length, you can do this:

var decoder = Encoding.UTF8.GetDecoder();
// +1 in case it includes a work-in-progress char from the previous buffer
char[] chars = decoder.GetMaxCharCount(bufferSize) + 1;
foreach (var byteSegment in bytes)
{
    int numChars = decoder.GetChars(byteSegment, 0, byteSegment.Length, chars, 0);
    Debug.WriteLine(new string(chars, 0, numChars));
}

If the buffers have different lengths:

var decoder = Encoding.UTF8.GetDecoder();
char[] chars = Array.Empty();
foreach (var byteSegment in bytes)
{
    // +1 in case it includes a work-in-progress char from the previous buffer
    int charsMinSize = decoder.GetMaxCharCount(bufferSize) + 1;
    if (chars.Length < charsMinSize)
        chars = new char[charsMinSize];
    int numChars = decoder.GetChars(byteSegment, 0, byteSegment.Length, chars, 0);
    Debug.WriteLine(new string(chars, 0, numChars));
}