Why does a GCE volume mount in a kubernetes pod cause a delay?

强颜欢笑 提交于 2019-12-02 06:42:12

问题


I have a kubernetes pod to which I attach a GCE persistent volume using a persistence volume claim. (For the even worse issue without a volume claim see: Mounting a gcePersistentDisk kubernetes volume is very slow)

When there is no volume attached, the pod starts in no time (max 2 seconds). But when the pod has a GCE persistent volume mount, the Running state is reached somewhere between 20 and 60 seconds. I was testing with different disk sizes (10, 200, 500 GiB) and multiple pod creations and the size does not seem to be correlated with the delay.

And this delay is not only happening in the beginning but also when rolling updates are performed with the replication controllers or when the code crashes during runtime.

Below I have the kubernetes specifications:

The replication controller

{
    "apiVersion": "v1",
    "kind": "ReplicationController",
    "metadata": {
        "name": "a1"
    },
    "spec": {
        "replicas": 1,
        "template": {
            "metadata": {
                "labels": {
                    "app": "a1"
                }
            },
            "spec": {
                "containers": [
                    {
                        "name": "a1-setup",
                        "image": "nginx",
                        "ports": [
                            {
                                "containerPort": 80
                            },
                            {
                                "containerPort": 443
                            }
                        ]
                    }
                ]
            }
        }
    }
}

The volume claim

{
    "apiVersion": "v1",
    "kind": "PersistentVolumeClaim",
    "metadata": {
        "name": "myclaim"
    },
    "spec": {
        "accessModes": [
            "ReadWriteOnce"
        ],
        "resources": {
            "requests": {
                "storage": "10Gi"
            }
        }
    }
}

And the volume

{
    "apiVersion": "v1",
    "kind": "PersistentVolume",
    "metadata": {
        "name": "mydisk",
        "labels": {
             "name": "mydisk"
        }
    },
    "spec": {
        "capacity": {
            "storage": "10Gi"
        },
        "accessModes": [
            "ReadWriteOnce"
        ],
        "gcePersistentDisk": {
            "pdName": "a1-drive",
            "fsType": "ext4"
        }
    }
}

Also


回答1:


GCE (along with AWS and OpenStack) must first attach a disk/volume to the node before it can be mounted and exposed to your pod. The time required for attachment is dependent on the cloud provider.

In the case of pods created by a ReplicationController, there is an additional detach operation that has to happen. The same disk cannot be attached to more than one node (at least not in read/write mode). Detaching and pod cleanup happen in a different thread than attaching. To be specific, Kubelet running on a node has to reconcile the pods it currently has (and the sum of their volumes) with the volumes currently present on the node. Orphaned volumes are unmounted and detached. If your pod was scheduled on a different node, it must wait until the original node detaches the volume.

The cluster eventually reaches the correct state, but it might take time for each component to get there. This is your wait time.



来源:https://stackoverflow.com/questions/34854472/why-does-a-gce-volume-mount-in-a-kubernetes-pod-cause-a-delay

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!