Argument list too long error for rm, cp, mv commands

Question

I have several hundred PDFs under a directory in UNIX. The names of the PDFs are really long (approx. 60 chars).

When I try to delete all PDFs together using the fol

Answer 1

I found that for extremely large lists of files (>1e6), these answers were too slow. Here is a solution using parallel processing in python. I know, I know, this isn't linux... but nothing else here worked.

(This saved me hours)

# delete files
import os as os
import glob
import multiprocessing as mp

directory = r'your/directory'
os.chdir(directory)


files_names = [i for i in glob.glob('*.{}'.format('pdf'))]

# report errors from pool

def callback_error(result):
    print('error', result)

# delete file using system command
def delete_files(file_name):
     os.system('rm -rf ' + file_name)

pool = mp.Pool(12)  
# or use pool = mp.Pool(mp.cpu_count())


if __name__ == '__main__':
    for file_name in files_names:
        print(file_name)
        pool.apply_async(delete_files,[file_name], error_callback=callback_error)