How to prevent rebuilding inside a draggable when child and feedback are the same

Question

Problem

I have a textfield inside a card that I want the user to be able to drag. So the child and the feedback should always be in the same state. However as soon