Refusing to split GroupedShuffleRangeTracker proposed split position is out of range

Question

I am sporadically getting the following errors:

W Refusing to split at \'\\x00\\x00\\x00\\x15\\xbc\\x19)b\\x00\\x01\': proposed split position i

Answer 1

This is not an error, it's part of normal operation of a pipeline. We should probably reduce its logging level to INFO and rephrase it, because it very frequently confuses people.

This message (rather obscurely) signals that Dataflow is trying to apply dynamic rebalancing, but there's no work that can be further subdivided.

I.e. your job is stuck doing something non-parallelizable on a small number of workers, while other workers are staying idle. To investigate this further, one would need to look at the code of your job and the Dataflow job id.