Spring - add a low priority multi threaded service (no impact to production performance)

Question

We have a Spring application, I want to add a service that will handle 10Ks IDs with multiple threads but will be as background process without impact production realtim

Answer 1

Here there is a nice tutorial about priority based task execution in Spring. Maybe this may help you in some ways.

This is a method of creating a configurable ¨heap¨ of task and always keep your main task in the top of the heap.

To sum up this process you should create a custom Task Executor. Firstly you need to create a ThreadPoolTaskExecutor bean with one method being overidden. The properties that should be modified are: CorePoolSize(initial number of threads), QueueCapacity(the number of threads waiting in the queue), and MaxPoolSize(maximum number of threads). With these parameters you can configure your applications limitations in order for this service not to impact the production performance.

 @Bean("CustomTaskExecutor")
  public TaskExecutor threadPoolTaskExecutor(
          @Value("${spring.async.core-pool-size}") int corePoolSize,
          @Value("${spring.async.max-pool-size}") int maxPoolSize,
          @Value("${spring.async.queue-capacity}") int queueCapacity) {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor() {

      @Override
      protected BlockingQueue<Runnable> createQueue(int queueCapacity) {
        return new PriorityBlockingQueue<Runnable>(queueCapacity);
      }

    };
    executor.setCorePoolSize(corePoolSize);
    executor.setMaxPoolSize(maxPoolSize);
    executor.setQueueCapacity(queueCapacity);
    return executor;
  }

After that,you need to make tasks with priorities that the task executor can understand. For that we would need to create two classes: 1) A custom class that implements Runnable interface that will be running the task 2) A wrapper class that extends FutureTask and implementsComparable interface, so that the task executor could understand the priority picking logic of the tasks

public class Task implements Runnable {
    private Consumer<Job> jobConsumer;
    private Job job;
      public Job getJob() {
        return this.job;
      }
      public Task(Consumer<Job> jobConsumer, Job job) {
        this.jobConsumer = jobConsumer;
        this.job = job;
      }
      @Override
      public void run() {
        this.jobConsumer.accept(job);
      }
    }

Then you have the FutureCustomTask class:

public class FutureCustomTask extends FutureTask<FutureCustomTask> implements Comparable<FutureCustomTask> {
private Task task;
public FutureCustomTask(Task task) {
    super(task, null);
    this.task = task;
  }
@Override
  public int compareTo(FutureCustomTask o) {
    return task.getJob().getPriority().compareTo(o.task.getJob().getPriority());
  }
}

For the execution the TaskExecutor needs to be Autowired. Then, you can create your Task object, wrap it inside FutureCustomTask, and pass it to TaskExecutor.The code should look like this:

@Autowired
private TaskExecutor taskExecutor;
@Autowired
private JobBusiness jobBusiness;
...
Task task = new Task(jobBusiness::performSomethingOn, job);
taskExecutor.execute(new FutureCustomTask(task));

Answer 2

I would like to throw some light on the question

what is a thread priority? According to java SE Docs

Every thread has a priority. Threads with higher priority are executed in preference to threads with lower priority.

Even though you create threads with priority it does not completely guarantee that threads with lower priority get executed first you may have to block the thread with lower priority until other threads are executed

For small java programs you can handle the thread execution by yourself but for larger programs, it's recommended either you use the Executor Framework from vanilla Java which is from the package java.util.concurrent or use the spring TaskExecutor.By using both frameworks you can execute tasks asynchronously that is executing them in the background as part of the main task.

Impact on Production:

The main task, for example, will be a call to your rest endpoint i.e /account and on calling the account endpoint you want to send welcome emails to customers which are a third party API call which can be executed asynchronously either using Executor Framework or Spring TaskExecutor on executing them asynchronously i.e as background process they will not have an impact on the current API but it will surely have an impact on the production server since you are running the threads within the same JVM and they share common memory. if there are a number of threads created and not destroyed .you server will surely go down.

So using Executor Framework or Spring TaskExecutor does not guarantee you that it will not affect your current production it will surely increase the performance of the rest API that is called. since it's executed asynchronously and on your other questions

can i use Executors.newCachedThreadPool()

yes if you have a number of the short-lived task such as updating a single column in a database or triggering a rest endpoint only once and it's not suitable for bulk loading or executing some backend job which updates 10000 records because it will create larger number of threads for each task and you will surely face memory problems.

Also, should I use Spring bean? because I need to create a pool on-demand/request so it seems not needed/wrong

yes you can ThreadPoolTaskExecutor use https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.html as per the docs Setting "queueCapacity" to 0 mimics Executors.newCachedThreadPool() .so you can use either Spring TaskExecutor or use Executor Framework from vanilla Java I personally recommend using Spring TaskExecutor which has more feature for a good start on using Spring you can refer the tutorial https://egkatzioura.com/2017/10/25/spring-and-async/ which is a good start

final verdict: if you are looking to execute the task only asynchronous or a background process you can use either use Executor Framework from java or Spring TaskExecutor but both will have an impact on production since they use the same JVM .if you do not want to impact production at all then I recommend creating separate spring boot app on a different server and make the database calls or service call from the new app and expose it as a rest endpoint and invoke these endpoints asynchronously from your main app using Spring Task Executor.

Spring Task Executor: https://egkatzioura.com/2017/10/25/spring-and-async/

Java Executor Framework : https://stackabuse.com/concurrency-in-java-the-executor-framework/

for using threads with low priority : https://medium.com/@daniyaryeralin/priority-based-thread-pooling-in-spring-framework-d74b91b51dcb

Sorry if the answer is too long :)