Multithreading best practices : constraining tasks newFixedThreadPool

Question

I want to launch a lot of tasks to run on a database of +-42Mio records. I want to run this in batches of 5000 records/time (results in 850 tasks). I also want to limit the

Answer 1

The first part using ExecutorService looks good:

...
FetcherRunner runner = new FetcherRunner(routes, start, stop);
executorService.submit(runner);

The part with Thread should not be there, I am assuming you have it there just to show how you had it before?

Update: Yes, you don't require the code after executorService.submit(runner), that is going to end up spawning a huge number of threads. If your objective is to wait for all submitted tasks to complete after the loop, then you can get a reference to Future when submitting tasks and wait on the Future, something like this:

ExecutorService executorService = Executors.newFixedThreadPool(16);
List> futures = ..;
 for (int j = 1; j < 900+ 1; j++) {
 int start = (j - 1) * 5000;
 int stop = (j) * 5000- 1;
 FetcherRunner runner = new FetcherRunner(routes, start, stop);
 futures.add(executorService.submit(runner));

}
for (Future future:futures){
    future.get(); //Do something with the results..
}