How to Change Kubernetes Node Status from “Ready” to “NotReady” by changing CPU Utilization or memory utilization or Disk Pressure?

问题

I have kubernetes cluster set up of 1 master and 1 worker node.For testing purpose , I increased CPU Utlization and memory utilization upto 100%, but stilling Node is not getting status "NotReady". I am testing Pressure Status .. How to change status flag of MemoryPressure to true or DiskPressure or PIDPressure to true

Here are my master node condtions :-

Conditions:

 Type                 Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----                 ------  -----------------                 ------------------                ------                       -------
  NetworkUnavailable   False   Wed, 27 Nov 2019 14:36:29 +0000   Wed, 27 Nov 2019 14:36:29 +0000   WeaveIsUp                    Weave pod has set this
  MemoryPressure       False   Thu, 28 Nov 2019 07:36:46 +0000   Fri, 22 Nov 2019 13:30:38 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure         False   Thu, 28 Nov 2019 07:36:46 +0000   Fri, 22 Nov 2019 13:30:38 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure          False   Thu, 28 Nov 2019 07:36:46 +0000   Fri, 22 Nov 2019 13:30:38 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready                True    Thu, 28 Nov 2019 07:36:46 +0000   Fri, 22 Nov 2019 13:30:48 +0000   KubeletReady                 kubelet is posting ready status

Here are pods info :-

Non-terminated Pods:         (8 in total)
  Namespace                  Name                                                                   CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                  ----                                                                   ------------  ----------  ---------------  -------------  ---
  kube-system                coredns-5644d7b6d9-dm8v7                                               100m (5%)     0 (0%)      70Mi (0%)        170Mi (2%)     22d
  kube-system                coredns-5644d7b6d9-mz5rm                                               100m (5%)     0 (0%)      70Mi (0%)        170Mi (2%)     22d
  kube-system                etcd-ip-172-31-28-186.us-east-2.compute.internal                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         22d
  kube-system                kube-apiserver-ip-172-31-28-186.us-east-2.compute.internal             250m (12%)    0 (0%)      0 (0%)           0 (0%)         22d
  kube-system                kube-controller-manager-ip-172-31-28-186.us-east-2.compute.internal    200m (10%)    0 (0%)      0 (0%)           0 (0%)         22d
  kube-system                kube-proxy-cw8vv                                                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         22d
  kube-system                kube-scheduler-ip-172-31-28-186.us-east-2.compute.internal             100m (5%)     0 (0%)      0 (0%)           0 (0%)         22d
  kube-system                weave-net-ct9zb                                                        20m (1%)      0 (0%)      0 (0%)           0 (0%)         22d
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                770m (38%)  0 (0%)
  memory             140Mi (1%)  340Mi (4%)
  ephemeral-storage  0 (0%)      0 (0%)

Here for worker node :-

Conditions:
  Type                 Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----                 ------  -----------------                 ------------------                ------                       -------
  NetworkUnavailable   False   Thu, 28 Nov 2019 07:00:08 +0000   Thu, 28 Nov 2019 07:00:08 +0000   WeaveIsUp                    Weave pod has set this
  MemoryPressure       False   Thu, 28 Nov 2019 07:39:03 +0000   Thu, 28 Nov 2019 07:00:00 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure         False   Thu, 28 Nov 2019 07:39:03 +0000   Thu, 28 Nov 2019 07:00:00 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure          False   Thu, 28 Nov 2019 07:39:03 +0000   Thu, 28 Nov 2019 07:00:00 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready                True    Thu, 28 Nov 2019 07:39:03 +0000   Thu, 28 Nov 2019 07:00:00 +0000   KubeletReady                 kubelet is posting ready status

回答1:

There are several ways of making a node to get into NotReady status, but not through Pods. When a Pod starts to consume too much memory kubelet will just kill that pod, to precisely protect the node.

I guess you want to test what happens when a node goes down, in that case you want to drain it. In other words, to simulate node issues, you should do:

kubectl drain NODE

Still, check on kubectl drain --help to see under what circumstances what happens.

EDIT

I tried, actually, accessing the node and running stress on the node directly, and this is what happened within 20 seconds:

root@gke-klusta-lemmy-3ce02acd-djhm:/# stress --cpu 16 --io 8 --vm 8 --vm-bytes 2G

Checking on the node:

$ kubectl get no -w | grep gke-klusta-lemmy-3ce02acd-djhm
gke-klusta-lemmy-3ce02acd-djhm   Ready    <none>   15d     v1.13.11-gke.14
gke-klusta-lemmy-3ce02acd-djhm   Ready   <none>   15d   v1.13.11-gke.14
gke-klusta-lemmy-3ce02acd-djhm   NotReady   <none>   15d   v1.13.11-gke.14
gke-klusta-lemmy-3ce02acd-djhm   NotReady   <none>   15d   v1.13.11-gke.14
gke-klusta-lemmy-3ce02acd-djhm   NotReady   <none>   15d   v1.13.11-gke.14

I am running pretty weak nodes. 1CPU@4GB RAM

回答2:

Just bring down kubelet service on the node that you want to be in NotReady status

来源：https://stackoverflow.com/questions/59083863/how-to-change-kubernetes-node-status-from-ready-to-notready-by-changing-cpu

标签

Kubernetes