How do I loop through a large dataset in python without getting a MemoryError?

前端 未结 2 2025
自闭症患者
自闭症患者 2021-02-08 02:47

I have a large series of raster datasets representing monthly rainfall over several decades. I\'ve written a script in Python that loops over each raster and does the following:

2条回答
  •  情深已故
    2021-02-08 03:32

    a quick way to "force" the garbage collector to clean the temporary loop-only objects is the del statement:

    for obj in list_of_obj:   
        data = obj.getData()  
        do_stuff(data)   
        del data 
    

    this forces the interpreter to delete and free the temporary objects. NOTE: this does not make sure the program does not leak or consume memory in other parts of the computation, it's just a quick check

提交回复
热议问题