Lazy Method for Reading Big File in Python?

前端 未结 12 1753
谎友^
谎友^ 2020-11-21 06:40

I have a very big file 4GB and when I try to read it my computer hangs. So I want to read it piece by piece and after processing each piece store the processed piece into an

12条回答
  •  醉话见心
    2020-11-21 07:21

    I think we can write like this:

    def read_file(path, block_size=1024): 
        with open(path, 'rb') as f: 
            while True: 
                piece = f.read(block_size) 
                if piece: 
                    yield piece 
                else: 
                    return
    
    for piece in read_file(path):
        process_piece(piece)
    

提交回复
热议问题