问题
We have 4 jobs that are running over 3 nodes with 4 slots per each,
On Flink 1.3.2 the jobs were evenly distributed per node. After upgrading to flink 1.5 , each job is running on a single node (with a carry over to another if there are no slots left)
Is there a way to return to an even distribution? The jobs are not evenly by load which cause some nodes to work harder than other.
回答1:
An answer I received from flink mailing list Re: Flink 1.5 job distribution over cluster nodes
Hi Shachar,
with Flink 1.5 we added resource elasticity. This means that Flink is now able to allocate new containers on a cluster management framework like Yarn or Mesos. Due to these changes (which also apply to the standalone mode), Flink no longer reasons about a fixed set of TaskManagers because if needed it will start new containers (does not work in standalone mode). Therefore, it is hard for the system to make any decisions about spreading slots belonging to a single job out across multiple TMs. It gets even harder when you consider that some jobs like yours might benefit from such a strategy whereas others would benefit from co-locating its slots. It gets even more complicated if you want to do scheduling wrt to multiple jobs which the system does not have full knowledge about because they are submitted sequentially. Therefore, Flink currently assumes that slots requests can be fulfilled by any TaskManager.
Cheers, Till
来源：https://stackoverflow.com/questions/51345472/flink-job-distribution-over-cluster-nodes