Celery CPU usage creeping up over time with many workers

Question

So I\'m trying to run a big web scraping job (6m+ websites) with Python + Celery + Redis.

I\'m running on a big box (ml.m5.16xlarge: 64 vCPU + 256 GB RAM) and I\'m no