Since I updated to the latest Xcode version 11.0 (11A420a) the debugging is extremely slow and cumbersome.
EDIT: After updating to 11.1 GM (11A1027
I had 238 unused breakpoints set. When I deleted them, it's fast again (even with diagnostics turned on.)