Sorting gigantic binary files with C#

Question

I have a large file of roughly 400 GB of size. Generated daily by an external closed system. It is a binary file with the following format:

byte[8]byte[4]byte[n         


        

Answer 1

If you can learn Erlang or Go, they could be very powerful and scale extremely well, as you have 24 threads. Utilize Async I/O. Merge Sort. And since you have 32GB of Ram, try to load as much as you can into RAM and sort it there then write back to disk.