How to split a large text file into smaller files with equal number of lines?

前端 未结 10 595
一整个雨季
一整个雨季 2020-11-22 16:42

I\'ve got a large (by number of lines) plain text file that I\'d like to split into smaller files, also by number of lines. So if my file has around 2M lines, I\'d like to

10条回答
  •  遇见更好的自我
    2020-11-22 17:13

    In case you just want to split by x number of lines each file, the given answers about split are OK. But, i am curious about no one paid attention to requirements:

    • "without having to count them" -> using wc + cut
    • "having the remainder in extra file" -> split does by default

    I can't do that without "wc + cut", but I'm using that:

    split -l  $(expr `wc $filename | cut -d ' ' -f3` / $chunks) $filename
    

    This can be easily added to your bashrc functions so you can just invoke it passing filename and chunks:

     split -l  $(expr `wc $1 | cut -d ' ' -f3` / $2) $1
    

    In case you want just x chunks without remainder in extra file, just adapt the formula to sum it (chunks - 1) on each file. I do use this approach because usually i just want x number of files rather than x lines per file:

    split -l  $(expr `wc $1 | cut -d ' ' -f3` / $2 + `expr $2 - 1`) $1
    

    You can add that to a script and call it your "ninja way", because if nothing suites your needs, you can build it :-)

提交回复
热议问题