Adding multi-threading possibility to a single-threaded all-files-in-directory iterator utility function

问题

I have a function that serially (single-threaded-ly) iterates through a directory of files, changing all tab indentation to three-space indentation.

I'm using it as my first attempt at multi-threading. (Am most of the way through Java Concurrency in Practice...surprised it's eight years old now.)

In order to keep it's current single-threaded functionality, but add in the additional possibility of multi-threading, I'm thinking of changing the function to accept an additional Executor parameter, where the original single-threaded function would now be a call to it, passing in a single threaded executor.

Is this an appropriate way to go about it?

回答1:

One way is as @Victor Sorokin suggests in his answer: wrap the processing of every file in a Runnable and then either submit to an Executor or just invoke run() from the main thread.

Another possibility is to always do the same wrapping in a Runnable and submit it to an always-given Executor.

Whether processing of each file is executed concurrently or not would depend on the given Executor's implementation.

For parallel processing, you could invoke your function passing it i.e. a ThreadPoolExecutor as an argument, whereas for sequential processing you could pass in a fake Executor, i.e. one that runs submitted tasks in the caller thread:

public class FakeExecutor implements Executor {

    @Override
    public void execute(Runnable task) {
        task.run();
    }
}

I believe this way is the most flexible approach.

回答2:

If you're using Java 8, I've found parallelStream to be about the easiest way to implement multi-threading

    List<File> files = Arrays.asList(getDirectoryContents());
    files.parallelStream().forEach( file -> processFile(file));

If you want to be able to change between single-threaded and multi-threaded, you could simply pass a boolean flag

    List<File> files = Arrays.asList(getDirectoryContents());
    if(multithreaded){
      files.parallelStream().forEach( file -> processFile(file));
    }else{
      files.stream().forEach(file -> processFile(file));
    }

I wish I could help with Java 7, but I went from Java 5 to 8 overnight. :) Java 8 is sooooooo worth it.

回答3:

Most straight-forward way:

(The most tricky part) Make sure code is thread-safe. Unfortunately, it's hard to give more concrete advice w/o seeing actual code in question;
Wrap code into Runnable\Callable (either anonymous class or explicit class which implements Runnable\Callable;

This way you'll be able either call your Runnable in main thread (single-threaded version) or pass it to an Executor (multi-threaded version).

回答4:

One of the ways to create a class implements Executor interface which will execute your code in the main thread. Like this:

public class FileProcessor implements Runnable {

    private final File file;

    public FileProcessor(File file) {
        this.file = file;
    }

    @Override
    public void run() {
        // do something with file
    }
}

public class DirectoryManager {

    private final Executor executor;

    public DirectoryManager() {
        executor = new Executor() {
            @Override
            public void execute(Runnable command) {
                command.run();
            }
        };
    }

    public DirectoryManager(int numberOfThreads) {
        executor = Executors.newFixedThreadPool(numberOfThreads);
    }

    public void process(List<File> files) {
        for (File file : files) {
            executor.execute(new FileProcessor(file));
        }
    }
}

and call it in your code like this

DirectoryManager directoryManager = new DirectoryManager();
directoryManager.process(lists);
// some other sync code

or this

DirectoryManager directoryManager = new DirectoryManager(5);
directoryManager.process(lists);
// some other async code

来源：https://stackoverflow.com/questions/29263326/adding-multi-threading-possibility-to-a-single-threaded-all-files-in-directory-i

标签

java

multithreading

java-7

executor