Reactor Schedulers keep running long after main thread is done?How to deal with this?

问题

I have a question on how to clean up the Scheduler worker threads while using Reactor 3

Flux.range(1, 10000)
.publishOn(Schedulers.newElastic("Y"))
.doOnComplete(() -> { 
    // WHAT should one do to ensure the worker threads are cleaned up
    logger.info("Shut down all Scheduler worker threads");
})
.subscribe(x -> logger.debug(x+ "**"));

What I see when I execute the above code is that once the main thread has finished running the worker thread(s) is/are still in WAITING state for some time.

sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)

Is there a way to control this ?i.e can they be disposed onComplete() ?I tried Schedulers.shutdownNow() and it doesn't help.

On the other hand when I do this I'm able to control the Scheduler disposal. Which is the preferred/advocated way ?

reactor.core.scheduler.Scheduler s = Schedulers.newElastic("X");
        Flux.range(1, 10000)
        .concatWith(Flux.empty())
        .publishOn(s)
        .doOnComplete(() -> {           
            s.dispose();
            logger.info("Shut down all Scheduler worker threads");
        })
        .subscribe(x -> logger.debug(x+ "**"));

回答1:

If you use Schedulers.new[Elastic|...], then it is your responsibility to keep track of the resulting Scheduler if you want to close it. Schedulers.shutdownNow() will only close the default schedulers used by the library when you're not explicit, like Schedulers.elastic() (notice no new prefix).

The best way to clean-up after all the operations have run is to use doFinally. This will asynchronously perform a cleanup callback after onError|onComplete|cancel events. Better ensure it is the last operator in the chain, although it tries to really execute last in all cases.

The only caveat is that it runs in the same thread as the previous operators, in other words the very thread you are trying to shutdown... a s.dispose() in the doFinally callback would shutdown the executor after its queue of tasks has been processed, so in this case there would be a slight delay before the thread disappears.

Here is a sample that dumps thread info, switches to the custom elastic thread and shuts it down in a doFinally (added filter and materialize to give a shorter log with a better view of how events play out):

@Test
public void schedulerFinallyShutdown() throws InterruptedException {
    ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
    Logger logger = Loggers.getLogger("foo");
    CountDownLatch latch = new CountDownLatch(1);
    reactor.core.scheduler.Scheduler s = Schedulers.newElastic("X");
    Flux.range(1, 10000)
        .publishOn(s)
        .concatWith(Flux.<Integer>empty().doOnComplete(() -> {
            for (ThreadInfo ti : threadMxBean.dumpAllThreads(true, true)) {
                System.out.println("last element\t" + ti.getThreadName() + " " + ti.getThreadState());
            }
        }))
        .doFinally(sig -> {
            s.dispose();
            logger.info("Shut down all Scheduler worker threads");
            latch.countDown();
        })
        .filter(x -> x % 1000 == 0)
        .materialize()
        .subscribe(x -> logger.info(x+ "**"));

    latch.await();
    for (ThreadInfo ti : threadMxBean.dumpAllThreads(true, true)) {
        System.out.println("after cleanup\t" + ti.getThreadName() + " " + ti.getThreadState());
    }
}

This prints out:

11:24:36.608 [X-2] INFO  foo - onNext(1000)**
11:24:36.611 [X-2] INFO  foo - onNext(2000)**
11:24:36.611 [X-2] INFO  foo - onNext(3000)**
11:24:36.612 [X-2] INFO  foo - onNext(4000)**
11:24:36.612 [X-2] INFO  foo - onNext(5000)**
11:24:36.612 [X-2] INFO  foo - onNext(6000)**
11:24:36.612 [X-2] INFO  foo - onNext(7000)**
11:24:36.613 [X-2] INFO  foo - onNext(8000)**
11:24:36.613 [X-2] INFO  foo - onNext(9000)**
11:24:36.613 [X-2] INFO  foo - onNext(10000)**
last element    X-2 RUNNABLE
last element    elastic-evictor-1 TIMED_WAITING
last element    Monitor Ctrl-Break RUNNABLE
last element    Signal Dispatcher RUNNABLE
last element    Finalizer WAITING
last element    Reference Handler WAITING
last element    main WAITING
11:24:36.626 [X-2] INFO  foo - onComplete()**
11:24:36.627 [X-2] INFO  foo - Shut down all Scheduler worker threads
after cleanup   Monitor Ctrl-Break RUNNABLE
after cleanup   Signal Dispatcher RUNNABLE
after cleanup   Finalizer WAITING
after cleanup   Reference Handler WAITING
after cleanup   main RUNNABLE

回答2:

You need to call blockLast() method on you flux to ensure it is completed. It is especially important if you are running you flux in parallel or in another thread than the main thread.

NB: You need to call publishOn higher on the chain, see the reference guide to see why the position of publishon is important.

reactor.core.scheduler.Scheduler s = Schedulers.newElastic("X");
Flux.range(1, 10000)
    .concatWith(Flux.empty())
    .publishOn(s)
    .doOnComplete(() -> {           
       logger.info("Shut down all Scheduler worker threads");
    }).blocklast();
s.dispose();

来源：https://stackoverflow.com/questions/47914755/reactor-schedulers-keep-running-long-after-main-thread-is-donehow-to-deal-with

标签

java

project-reactor