Multiline file read in Python

陌路散爱 提交于 2019-12-08 04:07:37

问题


I am looking for a method in Python which can read multiple lines from a file(10 lines at a time). I have already looked into readlines(sizehint), I tried to pass value 10 but doesn't read only 10 lines. It actually reads till end of the file(I have tried on the small file). Each line is 11 bytes long and each read should fetch me 10 lines each time. If less than 10 lines are found then return only those lines. My actual file contains more than 150K lines.

Any idea how I can achieve this?


回答1:


You're looking for itertools.islice():

with open('data.txt') as f:
    lines = []
    while True:
        line = list(islice(f, 10)) #islice returns an iterator ,so you convert it to list here.
        if line:                     
            #do something with current set of <=10 lines here
            lines.append(line)       # may be store it 
        else:
            break
    print lines    



回答2:


This should do it

def read10Lines(fp):
    answer = []
    for i in range(10):
        answer.append(fp.readline())
    return answer

Or, the list comprehension:

ten_lines = [fp.readline() for _ in range(10)]

In both cases, fp = open('path/to/file')




回答3:


Another solution which can get rid of the silly infinite loop in favor of a more familiar for loop relies on itertools.izip_longest and a small trick with iterators. The trick is that zip(*[iter(iterator)]*n) breaks iterator up into chunks of size n. Since a file is already generator-like iterator (as opposed to being sequence like), we can write:

from itertools import izip_longest
with open('data.txt') as f:
    for ten_lines in izip_longest(*[f]*10,fillvalue=None):
        if ten_lines[-1] is None:
           ten_lines = filter(ten_lines) #filter removes the `None` values at the end
        process(ten_lines) 



回答4:


from itertools import groupby, count
with open("data.txt") as f:
    groups = groupby(f, key=lambda x,c=count():next(c)//10)
    for k, v in groups:
        bunch_of_lines = list(v)
        print bunch_of_lines


来源:https://stackoverflow.com/questions/12790983/multiline-file-read-in-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!