How do I loop through a large dataset in python without getting a MemoryError?

前端 未结 2 2040
刺人心
刺人心 2021-02-08 02:41

I have a large series of raster datasets representing monthly rainfall over several decades. I\'ve written a script in Python that loops over each raster and does the following:

2条回答
  •  孤城傲影
    2021-02-08 03:19

    a quick way to "force" the garbage collector to clean the temporary loop-only objects is the del statement:

    for obj in list_of_obj:   
        data = obj.getData()  
        do_stuff(data)   
        del data 
    

    this forces the interpreter to delete and free the temporary objects. NOTE: this does not make sure the program does not leak or consume memory in other parts of the computation, it's just a quick check

提交回复
热议问题