Ensure Kubernetes Deployment has completed and all pods are updated and available

后端 未结 2 687
长发绾君心
长发绾君心 2021-02-04 04:36

The status of a deployment indicates that you can look at a deployments observedGeneration vs generation and when observedGeneration >= genera

2条回答
  •  渐次进展
    2021-02-04 04:47

    Update #2: Kubernetes 1.5 will ship with a much better version of kubectl rollout status and improve even further in 1.6, possibly replacing my custom solution/script laid out below.

    Update #1: I have turned my answer into a script hosted on Github which has received a small number of improving PRs by now.

    Original answer:

    First of all, I believe the kubectl command you got is not correct: It replaces all white spaces by commas but then tries to get the 4th field after separating by white spaces.

    In order to validate that a deployment (or upgrade thereof) made it to all pods, I think you should check whether the number of available replicas matches the number of desired replicas. That is, whether the AVAILABLE and DESIRED columns in the kubectl output are equal. While you could get the number of available replicas (the 5th column) through

    kubectl get deployment nginx | tail -n +2 | awk '{print $5}'

    and the number of desired replicas (2nd column) through

    kubectl get deployment nginx | tail -n +2 | awk '{print $2}'

    a cleaner way is to use kubectl's jsonpath output, especially if you want to take the generation requirement that the official documentation mentions into account as well.

    Here's a quick bash script I wrote that expects to be given the deployment name on the command line, waits for the observed generation to become the specified one, and then waits for the available replicas to reach the number of the specified ones:

    #!/bin/bash
    set -o errexit
    set -o pipefail
    set -o nounset
    
    deployment=
    
    get_generation() {
      get_deployment_jsonpath '{.metadata.generation}'
    }
    
    get_observed_generation() {
      get_deployment_jsonpath '{.status.observedGeneration}'
    }
    
    get_replicas() {
      get_deployment_jsonpath '{.spec.replicas}'
    }
    
    get_available_replicas() {
      get_deployment_jsonpath '{.status.availableReplicas}'
    }
    
    get_deployment_jsonpath() {
      local readonly _jsonpath="$1"
    
      kubectl get deployment "${deployment}" -o "jsonpath=${_jsonpath}"
    }
    
    if [[ $# != 1 ]]; then
      echo "usage: $(basename $0) " >&2
      exit 1
    fi
    
    readonly deployment="$1"
    
    readonly generation=$(get_generation)
    echo "waiting for specified generation ${generation} to be observed"
    while [[ $(get_observed_generation) -lt ${generation} ]]; do
      sleep .5
    done
    echo "specified generation observed."
    
    readonly replicas="$(get_replicas)"
    echo "specified replicas: ${replicas}"
    
    available=-1
    while [[ ${available} -ne ${replicas} ]]; do
      sleep .5
      available=$(get_available_replicas)
      echo "available replicas: ${available}"
    done
    
    echo "deployment complete."
    

提交回复
热议问题