Difficulty reading large file into byte array

问题

I have a very large BMP file that I have to read in all at once because I need to reverse the bytes when writing it to a temp file. This BMP is 1.28GB, and I'm getting the "Out of memory" error. I can't read it completely (using ReadAllBytes) or using a buffer into a binary array because I can't initialize an array of that size. I also can't read it into a List (which I could then Reverse()) using a buffer because halfway through it runs out of memory.

So basically the question is, how do I read a very large file backwards (ie, starting at LastByte and ending at FirstByte) and then write that to disk?

Bonus: when writing the reversed file to disk, do not write the last 54 bytes.

回答1:

With a StreamReader object, you can Seek (place the "cursor") to any particular byte, so you can use that to go over the entire file's contents in reverse.

Example:

const int bufferSize = 1024;
string fileName = 'yourfile.txt';

StreamReader myStream = new StreamReader(fileName);
myStream.BaseStream.Seek(bufferSize, SeekOrigin.End);

char[] bytes = new char[bufferSize];
while(myStream.BaseStream.Position > 0)
{
    bytes.Initialize();
    myStream.BaseStream.Seek(bufferSize, SeekOrigin.Current);
    int bytesRead = myStream.Read(bytes, 0, bufferSize);
}

回答2:

You can not normally handle so big files in .NET, due the implied memory limit for CLR applications and collections inside them neither for 32 nor for 64 platform.

For this you can use Memory Mapped File, to read a file directly from the disk, without loading it into the memory. One time memory mapping created move the reading pointer to end of the file and read backwards.

Hope this helps.

回答3:

You can use Memory Mapped Files.

http://msdn.microsoft.com/en-us/library/vstudio/dd997372%28v=vs.100%29.aspx

Also, you can use FileStream and positioning on necessary position by stream.Seek(xxx, SeekOrigin.Begin) (relative position) or Position property (absolute position).

来源：https://stackoverflow.com/questions/15874755/difficulty-reading-large-file-into-byte-array

标签

filestream

large-files