How do I read a large csv file with pandas?

后端 未结 15 1909
隐瞒了意图╮
隐瞒了意图╮ 2020-11-21 07:12

I am trying to read a large csv file (aprox. 6 GB) in pandas and i am getting a memory error:

MemoryError                               Traceback (most recen         


        
15条回答
  •  花落未央
    2020-11-21 07:27

    In case someone is still looking for something like this, I found that this new library called modin can help. It uses distributed computing that can help with the read. Here's a nice article comparing its functionality with pandas. It essentially uses the same functions as pandas.

    import modin.pandas as pd
    pd.read_csv(CSV_FILE_NAME)
    

自定义标题
段落格式
字体
字号
代码语言
提交回复
热议问题