问题
Based on no. of partitions returned can the maxWorkers changed dynamically(at runtime) when using DeployerPartitionHandler
Regards, Balu
UPDATE Please find my use case. The batch execution starts for a normal business day with maxWorkers as "4" and partition returned as 40 but suddenly the load increases and that particular run of the batch returned 4 times the normal partitions (160). Here how do I increase the maxWorkers (say 16). Also if I start the batch always with a very high limit for maxWorkers (say 16), I believe the workers for a normal load can also hit 16 as each partition is processed by a worker, right ?
回答1:
They cannot. The idea around maxWorkers
is really to set a high water mark that the app shouldn't go past. The dynamic aspect of the number of workers really is expected to come from the number of partitions. If you provide more detail in your question as to the specific use case, we can see if that still makes sense.
Update based on use case:
In your specific use case, you would need to have a listener (probably a JobExecutionListener
) that would determine how many the max should be. What you have described is that you want a "regular max" and a "high load max". The listener would run before the step in question and set the max value in the ExecutionContext
. Then, if the DeployerPartitionHandler
is step scoped, you can inject the value from the ExecutionContext
into the handler when that step executes.
来源:https://stackoverflow.com/questions/64560583/dynamically-change-maxworkers-with-deployerpartitionhandler