C# - Binary reader in Big Endian?

Question

I\'m trying to improve my understanding of the STFS file format by using a program to read all the different bits of information. Using a website with a reference of which o

Answer 1

I'm not usually one to answer my own questions, but I've accomplished exactly what I wanted with some simple code:

class BinaryReader2 : BinaryReader { 
    public BinaryReader2(System.IO.Stream stream)  : base(stream) { }

    public override int ReadInt32()
    {
        var data = base.ReadBytes(4);
        Array.Reverse(data);
        return BitConverter.ToInt32(data, 0);
    }

    public Int16 ReadInt16()
    {
        var data = base.ReadBytes(2);
        Array.Reverse(data);
        return BitConverter.ToInt16(data, 0);
    }

    public Int64 ReadInt64()
    {
        var data = base.ReadBytes(8);
        Array.Reverse(data);
        return BitConverter.ToInt64(data, 0);
    }

    public UInt32 ReadUInt32()
    {
        var data = base.ReadBytes(4);
        Array.Reverse(data);
        return BitConverter.ToUInt32(data, 0);
    }

}

I knew that's what I wanted, but I didn't know how to write it. I found this page and it helped: http://www.codekeep.net/snippets/870c4ab3-419b-4dd2-a950-6d45beaf1295.aspx

Answer 2

In my opinion, you want to be careful doing this. The reason one would want to Convert from BigEndian to LittleEndian is if the bytes being read are in BigEndian and the OS calculating against them is operating in LittleEndian.

C# isn't a window only language anymore. With ports like Mono, and also other Microsoft Platforms like Windows Phone 7/8, Xbox 360/Xbox One, Windwos CE, Windows 8 Mobile, Linux With MONO, Apple with MONO, etc. It is quite possible the operating platform could be in BigEndian, in which case you'd be screwing yourself if you converted the code without doing any checks.

The BitConverter already has a field on it called "IsLittleEndian" you can use this to determine if the operating environment is in LittleEndian or not. Then you can do the reversing conditionally.

As such, I actually just wrote some byte[] extensions instead of making a big class:

    /// <summary>
    /// Get's a byte array from a point in a source byte array and reverses the bytes. Note, if the current platform is not in LittleEndian the input array is assumed to be BigEndian and the bytes are not returned in reverse order
    /// </summary>
    /// <param name="byteArray">The source array to get reversed bytes for</param>
    /// <param name="startIndex">The index in the source array at which to begin the reverse</param>
    /// <param name="count">The number of bytes to reverse</param>
    /// <returns>A new array containing the reversed bytes, or a sub set of the array not reversed.</returns>
    public static byte[] ReverseForBigEndian(this byte[] byteArray, int startIndex, int count)
    {
        if (BitConverter.IsLittleEndian)
            return byteArray.Reverse(startIndex, count);
        else
            return byteArray.SubArray(startIndex, count);

    }

    public static byte[] Reverse(this byte[] byteArray, int startIndex, int count)
    {
        byte[] ret = new byte[count];
        for (int i = startIndex + (count - 1); i >= startIndex; --i)
        {
            byte b = byteArray[i];
            ret[(startIndex + (count - 1)) - i] = b;
        }
        return ret;
    }

    public static byte[] SubArray(this byte[] byteArray, int startIndex, int count)
    {
        byte[] ret = new byte[count];
        for (int i = 0; i < count; ++i)            
            ret[0] = byteArray[i + startIndex];
        return ret;
    }

So imagine this example code:

byte[] fontBytes = byte[240000]; //some data loaded in here, E.G. a TTF TrueTypeCollection font file. (which is in BigEndian)

int _ttcVersionMajor = BitConverter.ToUint16(fontBytes.ReverseForBigEndian(4, 2), 0);

//output
_ttcVersionMajor = 1 //TCCHeader is version 1

Answer 3

A mostly-complete (for my purposes) drop-in replacement for BinaryReader that handles endianness correctly, unlike most of these answers. By default it works exactly like BinaryReader, but can be constructed to read in the required endianness. Additionally the Read<Primitive> methods are overloaded to allow you to specify the endianness to read a particular value in - useful in the (unlikely) scenario that you're dealing with a stream of mixed LE/BE data.

public class EndiannessAwareBinaryReader : BinaryReader
{
    public enum Endianness
    {
        Little,
        Big,
    }

    private readonly Endianness _endianness = Endianness.Little;

    public EndiannessAwareBinaryReader(Stream input) : base(input)
    {
    }

    public EndiannessAwareBinaryReader(Stream input, Encoding encoding) : base(input, encoding)
    {
    }

    public EndiannessAwareBinaryReader(Stream input, Encoding encoding, bool leaveOpen) : base(input, encoding, leaveOpen)
    {
    }

    public EndiannessAwareBinaryReader(Stream input, Endianness endianness) : base(input)
    {
        _endianness = endianness;
    }

    public EndiannessAwareBinaryReader(Stream input, Encoding encoding, Endianness endianness) : base(input, encoding)
    {
        _endianness = endianness;
    }

    public EndiannessAwareBinaryReader(Stream input, Encoding encoding, bool leaveOpen, Endianness endianness) : base(input, encoding, leaveOpen)
    {
        _endianness = endianness;
    }

    public override short ReadInt16() => ReadInt16(_endianness);

    public override int ReadInt32() => ReadInt32(_endianness);

    public override long ReadInt64() => ReadInt64(_endianness);

    public override ushort ReadUInt16() => ReadUInt16(_endianness);

    public override uint ReadUInt32() => ReadUInt32(_endianness);

    public override ulong ReadUInt64() => ReadUInt64(_endianness);

    public short ReadInt16(Endianness endianness) => BitConverter.ToInt16(ReadForEndianness(sizeof(short), endianness));

    public int ReadInt32(Endianness endianness) => BitConverter.ToInt32(ReadForEndianness(sizeof(int), endianness));

    public long ReadInt64(Endianness endianness) => BitConverter.ToInt64(ReadForEndianness(sizeof(long), endianness));

    public ushort ReadUInt16(Endianness endianness) => BitConverter.ToUInt16(ReadForEndianness(sizeof(ushort), endianness));

    public uint ReadUInt32(Endianness endianness) => BitConverter.ToUInt32(ReadForEndianness(sizeof(uint), endianness));

    public ulong ReadUInt64(Endianness endianness) => BitConverter.ToUInt64(ReadForEndianness(sizeof(ulong), endianness));

    private byte[] ReadForEndianness(int bytesToRead, Endianness endianness)
    {
        var bytesRead = ReadBytes(bytesToRead);

        if ((endianness == Endianness.Little && !BitConverter.IsLittleEndian)
            || (endianness == Endianness.Big && BitConverter.IsLittleEndian))
        {
            Array.Reverse(bytesRead);
        }

        return bytesRead;
    }
}

Answer 4

I'm not familiar with STFS, but changing endianess is relatively easy. "Network Order" is big endian, so all you need to do is translate from network to host order.

This is easy because there's already code that does that. Look at IPAddress.NetworkToHostOrder, as explained here: ntohs() and ntohl() equivalent?

Answer 5

IMHO a slightly better answer as it doesn't require a different class to be newed-up, makes the big-endian calls obvious and allows big- and little-endian calls to be mixed in the stream.

public static class Helpers
{
  // Note this MODIFIES THE GIVEN ARRAY then returns a reference to the modified array.
  public static byte[] Reverse(this byte[] b)
  {
    Array.Reverse(b);
    return b;
  }

  public static UInt16 ReadUInt16BE(this BinaryReader binRdr)
  {
    return BitConverter.ToUInt16(binRdr.ReadBytesRequired(sizeof(UInt16)).Reverse(), 0);
  }

  public static Int16 ReadInt16BE(this BinaryReader binRdr)
  {
    return BitConverter.ToInt16(binRdr.ReadBytesRequired(sizeof(Int16)).Reverse(), 0);
  }

  public static UInt32 ReadUInt32BE(this BinaryReader binRdr)
  {
    return BitConverter.ToUInt32(binRdr.ReadBytesRequired(sizeof(UInt32)).Reverse(), 0);
  }

  public static Int32 ReadInt32BE(this BinaryReader binRdr)
  {
    return BitConverter.ToInt32(binRdr.ReadBytesRequired(sizeof(Int32)).Reverse(), 0);
  }

  public static byte[] ReadBytesRequired(this BinaryReader binRdr, int byteCount)
  {
    var result = binRdr.ReadBytes(byteCount);

    if (result.Length != byteCount)
      throw new EndOfStreamException(string.Format("{0} bytes required from stream, but only {1} returned.", byteCount, result.Length));

    return result;
  }
}

Answer 6

Checking the source of truth (the source code) I see BinaryWriter as writing data in big endian format. In BinaryReader, it is assuming big endian format and reading the data appropriately into an int, which should work regardless of CPU architecture. You should be able to write on big or little endian and read on the other CPU architecture without additional code. It's possible this is new as of .NET core...

So if you source file / stream is in big endian, you should be able to use BinaryReader to read the data without extra code.

Source:

https://github.com/microsoft/referencesource/blob/master/mscorlib/system/io/binarywriter.cs#L279

https://github.com/microsoft/referencesource/blob/master/mscorlib/system/io/binaryreader.cs#L173

https://github.com/microsoft/referencesource/blob/master/mscorlib/system/io/memorystream.cs#L253

Please someone chime in if I am wrong or missed something.