I have a recursive function for moving some circles on a canvas. Overed circle is enlarged (zoom in) and all the other circles is pushed away. Pushed circles push other circles
It doesn't surprise this overflows because the algorithm grows the stack as it iterates but the exit condition is unpredictable, actions are not tightly localized (they have knock-on effects to nearby circles), so processing time will be chaotic.
I would reconsider the algorithm. Consider finding the two closest circles, if these are further apart than a given threshold apart, abort. Otherwise move them apart a little and repeat.