Mapping a flat text file

时光总嘲笑我的痴心妄想 提交于 2019-12-08 07:14:35

问题


In a text file, lines are detected by \n at the end of each line. For this purpose, it is necessary to read the entire file, and this is a big problem for large files (say 2GB). I am looking for a method to read a single line without walking through the entire file (though I know it should be a complicated process).

  1. The first way I know is to use fseek() with offset; but it is not practical.
  2. Creating a flat file of key/value; but I am not sure if there is a way to avoid loading the entire into RAM (it should be something like reading an array in php).
  3. Alternatively, can we make some numbers at the beginning of each line to be read. I mean, is it possible to read the first digits at the beginning of the line by skipping the line contents (going to the next line).

    768| line content is here
    769| another line
    770| something
    

If reading only the first digits, the total data which should be read is not much even for large files.


回答1:


Do you need to read specific lines that can be indexed on line number?. If so just do a binary search. Read (say) 200 characters in the middle of the file to find out a line number. Then repeat in either of the halves until you get to the right line.




回答2:


I think there are no simple way to do what you want. Records have variable length and no length could be determined in advance, right?

If file is always the same (or at least not modified frequently), I'd put it to database, or at least create index file (record number: offset) and use that fseek()



来源:https://stackoverflow.com/questions/7709908/mapping-a-flat-text-file

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!