Parallelizing recursion in a for-loop using readdir

Question

I\'d like to parallelize a C-program which recursively calculates the size of a directory and its sub-directories, using OpenMP and C.

My issue is, that when I get i

Answer 1

You need a modern compiler with support for OpenMP tasks, which removes Visual C++ from the equation. Provided you are using such a compiler, all you need to do is to turn the recursive invocations to calculate_folder_size() into OpenMP tasks:

int calculate_folder_size(const char *path) {
    struct stat sb;

    if (S_ISREG(sb.st_mode)) { // if it's a file, not a directory (base case)
        return sb.st_size;
    }

    DIR *folder = opendir(path);

    struct dirent *element;
    size_t size = 4096;

    for (element = readdir(folder); element != NULL; element = readdir(folder)) {
        //(...)
        if (element->d_type == DT_DIR) {
            // Make sure the task receives a copy of the path
            char *priv_name = strdup(name); // (1)
            // recursive call of calculate_folder_size
            //               (2)
            #pragma omp task shared(size) firstprivate(priv_name)
            {
                // (3)
                #pragma omp atomic update
                size += calculate_folder_size(priv_name);
                free(priv_name); // (4)
            }
        } else {
            //(...)
        }
    }
    // (5)
    #pragma omp taskwait

    closedir(folder);
    return size;
}

The important parts here are:

1. You need to pass the task a name parameter that will live and retain its value until the task gets executed, which could be any time in the future. Therefore, you need to make a copy of name, e.g., using strdup(3).

2. The task should remember the current value of priv_name since it will change during the next iteration of the loop. Therefore, the firstprivate treatment of priv_name. It also needs to be able to modify size in the parent context, hence shared for it.

3. Since all tasks are updating the same size variable in the parent scope, the access needs to be protected with atomic update.

4. The private name is no longer needed and must be disposed.

5. The parent task should wait for all child tasks to finish first before returning size.

This function must be called from within a parallel region in order to do its job in parallel:

int size;

#pragma omp parallel
#pragma omp single
size = calculate_folder_size("/some/path");

It might be a good idea to limit the depth of parallelism at which things still run in parallel. I leave it to you to figure it out how :)