Starvation of Akka actors who participate in a sequence process

问题

Bisuness Logic

We have the following business logic to accomplish:

1 million times do:

download the file in index i
unzip the file
extract some info from the file
delete the file

Current Akka solution

The Akka solution that we have at the moment creates 1 million actors who are responsible for one file to download and once they are done they create an actor to take care of steps 2,3,4.

The problem

Once we run the process, we came across the situation where the Akka gives top priority to the download actors, and the rest of the actors are beeing in starvation mode.

We know that as the machine disk is getting full because the download actors are constantly downloading, but the other actors don't get a chance to scan and delete the files.

Questions

Is there a way to force Akka not to starve the actors down the actors chain?
Is there a way to tell a download actor to wait until it get some notification that it can continue (e.g. no more than parallel 1000 files in the disk)

Thanks.

回答1:

Use different dispatchers for the two types of actor:

In your config you can define a separate dispatcher as (for example):

my-dispatcher {
  type = Dispatcher
  executor = "thread-pool-executor"
  thread-pool-executor {
    fixed-pool-size = 32
  }
  throughput = 100
}

And then you can assign that to a specific actor at creation:

val myActor = context.actorOf(Props[MyActor].withDispatcher("my-dispatcher"), "myactor1")

Dispatchers are, effectively, thread-pools. Separating the two guarantees that the slow, blocking operations don't starve the other. This approach, in general, is referred to as bulk-heading, because the idea is that if a part of the app fails, the rest remains responsive.

For more info, see the documentation

来源：https://stackoverflow.com/questions/44668898/starvation-of-akka-actors-who-participate-in-a-sequence-process

标签

java

akka

actor

starvation