external-sorting | 易学教程

Efficiently reading a very large text file in C++

阅读更多关于 Efficiently reading a very large text file in C++

问题 I have a very large text file(45GB). Each line of the text file contains two space separated 64bit unsigned integers as shown below. 4624996948753406865 10214715013130414417 4305027007407867230 4569406367070518418 10817905656952544704 3697712211731468838 ... ... I want to read the file and perform some operations on the numbers. My Code in C++: void process_data(string str) { vector<string> arr; boost::split(arr, str, boost::is_any_of(" \n")); do_some_operation(arr); } int main() { unsigned

External Sort Java

阅读更多关于 External Sort Java

问题 Is there a specific reason why Java does not have an in-built external sort algorithm implemented ? 回答1: Because the JDK contains only the mostly used components. It is the same thing with any external framework content. Why isn't it directly built-in ? Simply because it doesn't need to be built-in. And because it's not developed by the same people. But still you can use an external framework, or a library which will help you with that. Resources : code.google.com - externalsortinginjava 回答2:

java.io.FileNotFoundException: access denied even though I've putted permissions in AndroidManifest

阅读更多关于 java.io.FileNotFoundException: access denied even though I've putted permissions in AndroidManifest

问题 I need again your help please!! I've an android application that write/read files to/from External memory. I've written all riquired permissions in AndroidManifest but I still get an error access denied. Hier my code: private static File convertStreamToFile(InputStream is) throws IOException { String dir = Environment.getExternalStorageDirectory().getAbsolutePath(); // I have tried this one too but it didn't work!! // File root = new File (Environment.getExternalStorageDirectory(),"ekg_daten

merging N sorted files using K way merge

阅读更多关于 merging N sorted files using K way merge

问题 There is decent literature about merging sorted files or say merging K sorted files. They all work on the theory that first element of each file is put in a Heap, then until the heap is empty poll that element, get another from the file from where this element was taken. This works as long as one record of each file can be put in a heap. Now let us say I have N sorted files but I can only bring K records in the heap and K < N and let us say N = Kc where "c" is the multiplier implying that N

Designing an external memory sorting algorithm

阅读更多关于 Designing an external memory sorting algorithm

问题 If I have a very large list stored in external memory that needs to be sorted. Asumimg this list is too large for internal memory, what major factors should be considered in designing an external sorting algorithm? 回答1: Before you go building your own external sort, you might look at the tools your operating system supplies. Windows has SORT.EXE, which works well enough on some text files, although it has ... idiosyncrasies. The GNU sort, too, works pretty well. You could give either of those

Designing an external memory sorting algorithm

阅读更多关于 Designing an external memory sorting algorithm

If I have a very large list stored in external memory that needs to be sorted. Asumimg this list is too large for internal memory, what major factors should be considered in designing an external sorting algorithm? Before you go building your own external sort, you might look at the tools your operating system supplies. Windows has SORT.EXE, which works well enough on some text files, although it has ... idiosyncrasies. The GNU sort, too, works pretty well. You could give either of those a try on a subset of your data to see if they'll do what you need. Otherwise . . . The external sort is a

How external merge sort algorithm works?

阅读更多关于 How external merge sort algorithm works?

I'm trying to understand how external merge sort algorithm works (I saw some answers for same question, but didn't find what I need). I'm reading the book "Analysis Of Algorithms" by Jeffrey McConnell and I'm trying to implement the algorithm described there. For example, I have input data: 3,5,1,2,4,6,9,8,7 , and I can load only 4 numbers into memory. My first step is read the input file in 4-number chunks, sort them in memory and write one to file A and next to file B. I got: A:[1,2,3,5][7] B:[4,6,8,9] Now my question how can I merge chunks from these files to the bigger ones if they will

Efficiently reading a very large text file in C++

阅读更多关于 Efficiently reading a very large text file in C++

I have a very large text file(45GB). Each line of the text file contains two space separated 64bit unsigned integers as shown below. 4624996948753406865 10214715013130414417 4305027007407867230 4569406367070518418 10817905656952544704 3697712211731468838 ... ... I want to read the file and perform some operations on the numbers. My Code in C++: void process_data(string str) { vector<string> arr; boost::split(arr, str, boost::is_any_of(" \n")); do_some_operation(arr); } int main() { unsigned long long int read_bytes = 45 * 1024 *1024; const char* fname = "input.txt"; ifstream fin(fname, ios::in

How external merge sort algorithm works?

阅读更多关于 How external merge sort algorithm works?

问题 I'm trying to understand how external merge sort algorithm works (I saw some answers for same question, but didn't find what I need). I'm reading the book "Analysis Of Algorithms" by Jeffrey McConnell and I'm trying to implement the algorithm described there. For example, I have input data: 3,5,1,2,4,6,9,8,7 , and I can load only 4 numbers into memory. My first step is read the input file in 4-number chunks, sort them in memory and write one to file A and next to file B. I got: A:[1,2,3,5][7]