multiprocessing returns “too many open files” but using `with…as` fixes it. Why?

Question

I was using this answer in order to run parallel commands with multiprocessing in Python on a Linux box.

My code did something like:

import multiproce         


        

Answer 1

This can happen when you use numpy.load too, make sure close those files too, or avoid using it and use pickle or torch.save torch.load etc.