问题
I am trying to test the HPA (horizontal pod autoscaling) in my Kubernetes cluster. Heapster is up and running and I think it works well since I'm able to see metrics in Grafana. Also the DNS addon is working perfectly.
Looking inside the HPA I can see the error "failed to get CPU consumption and request: failed to unmarshall heapster response: json: cannot unmarshal object into Go value of type []v1alpha1.PodMetrics"
$ kubectl describe hpa php-apache
Name: php-apache
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Wed, 10 Aug 2016 13:01:47 +0200
Reference: Deployment/php-apache
Target CPU utilization: 50%
Current CPU utilization: <unset>
Min replicas: 1
Max replicas: 10
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
36s 6s 3 {horizontal-pod-autoscaler } Warning FailedGetMetrics failed to get CPU consumption and request: failed to unmarshall heapster response: json: cannot unmarshal object into Go value of type []v1alpha1.PodMetrics
36s 6s 3 {horizontal-pod-autoscaler } Warning FailedComputeReplicas failed to get CPU utilization: failed to get CPU consumption and request: failed to unmarshall heapster response: json: cannot unmarshal object into Go value of type []v1alpha1.PodMetrics
The logs from Heapster says:
$ kubectl logs --namespace=kube-system heapster-vf9h9 -c heapster
I0810 06:38:17.178683 1 heapster.go:66] /heapster --source=kubernetes:https://kubernetes.default --sink=influxdb:http://monitoring-influxdb:8086
I0810 06:38:17.178947 1 heapster.go:67] Heapster version 1.2.0-beta.0
I0810 06:38:17.179091 1 configs.go:60] Using Kubernetes client with master "https://kubernetes.default" and version v1
I0810 06:38:17.179100 1 configs.go:61] Using kubelet port 10255
E0810 06:40:24.467817 1 influxdb.go:217] issues while creating an InfluxDB sink: failed to ping InfluxDB server at "monitoring-influxdb:8086" - Get http://monitoring-influxdb:8086/ping: dial tcp 192.168.112.71:8086: getsockopt: connection timed out, will retry on use
I0810 06:40:24.467853 1 influxdb.go:231] created influxdb sink with options: host:monitoring-influxdb:8086 user:root db:k8s
I0810 06:40:24.467883 1 heapster.go:96] Starting with InfluxDB Sink
I0810 06:40:24.467892 1 heapster.go:96] Starting with Metric Sink
I0810 06:40:24.481233 1 heapster.go:175] Starting heapster on port 8082
I0810 06:41:05.129926 1 influxdb.go:209] Created database "k8s" on influxDB server at "monitoring-influxdb:8086"
I0810 11:02:45.357708 1 handlers.go:190] No metrics for pod default/php-apache-1272773798-rm4i0
I0810 11:02:45.426775 1 handlers.go:190] No metrics for pod default/php-apache-1272773798-rm4i0
My cluster has 2 nodes and the master node is also a node. More info:
$ kubectl cluster-info
Kubernetes master is running at http://10.1.1.46:8080
Heapster is running at http://10.1.1.46:8080/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at http://10.1.1.46:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at http://10.1.1.46:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
monitoring-grafana is running at http://10.1.1.46:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.0", GitCommit:"283137936a498aed572ee22af6774b6fb6e9fd94", GitTreeState:"clean", BuildDate:"2016-07-01T19:26:38Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4", GitCommit:"dd6b458ef8dbf24aff55795baa68f83383c9b3a9", GitTreeState:"clean", BuildDate:"2016-08-01T16:38:31Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Is it a problem between the Heapster version and the Kubernetes version? I did the git clone of Heapster today, so it's the most recent release.
回答1:
It seems that you are using an incompatible version of Heapster. Please try with version 1.1.0, as defined here: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/cluster-monitoring/influxdb/heapster-controller.yaml
来源：https://stackoverflow.com/questions/38874145/autoscaling-hpa-failed-to-get-cpu-consumption-cannot-unmarshal-object-into-go