kubernetes scheduling for expensive resources
问题 We have a Kubernetes cluster. Now we want to expand that with GPU nodes (so that would be the only nodes in the Kubernetes cluster that have GPUs). We'd like to avoid Kubernetes to schedule pods on those nodes unless they require GPUs. Not all of our pipelines can use GPUs. The absolute majority are still CPU-heavy only. The servers with GPUs could be very expensive (for example, Nvidia DGX could be as much as $150/k per server). If we just add DGX nodes to Kubernetes cluster, then Kubernetes