Websocket returns 500 on client and 101 on server

问题

We try to implement WebSocket on the Kubernetes cluster using nginx-ingress controller.

ingress.yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/proxy-read-timeout: "7200"
    ingress.kubernetes.io/proxy-send-timeout: "7200"
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
  generation: 2
  labels:
    app: websocket
    app.kubernetes.io/managed-by: Helm
    chart: websocket-0.2.2693
    release: websocket
  name: websocket
  namespace: %NAME_SPACE%
spec:
  rules:
  - host: %HOST_NAME%
    http:
      paths:
      - backend:
          serviceName: websocket
          servicePort: 443
        path: /
      - backend:
          serviceName: websocket
          servicePort: 443
        path: /socket.io
status:
  loadBalancer:
    ingress:
    - ip: X.X.X.X
    - ip: Y.Y.Y.Y

service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: websocket
    meta.helm.sh/release-namespace: %NAME_SPACE%
  creationTimestamp: "2020-04-27T20:58:28Z"
  labels:
    app: websocket
    app.kubernetes.io/managed-by: Helm
    chart: websocket-0.2.2723
    release: websocket
  name: websocket
  namespace: %NAME_SPACE%
  resourceVersion: "2916073"
  selfLink: /api/v1/namespaces/%NAME_SPACE%/services/websocket
  uid: e4c08a00-6824-4e16-a3fa-cace0c9be519
spec:
  clusterIP: 10.0.3.45
  ports:
  - name: websocket
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: websocket
    release: websocket
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

deployment.yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "16"
    meta.helm.sh/release-name: websocket
  labels:
    app: websocket
    release: websocket
  name: websocket
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: websocket
      release: websocket
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    spec:
      containers:
        image: %NAME_SPACE%.azurecr.io/websocket:2723
        imagePullPolicy: Always
        name: websocket
        ports:
        - containerPort: 443
          name: websocket
          protocol: TCP
        resources:
          limits:
            cpu: 1500m
            memory: 1Gi
          requests:
            cpu: 250m
            memory: 64Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

When getting the log from nginx-ingress pod we noticed that the status code is 101 which means it's working.

kubectl logs %POD_NAME% -n nginx-ingress --since 1m | grep websocket

Output:

[11/May/2020:12:47:29 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 101 91

However, The client return 500 :

Error during WebSocket handshake: Unexpected response code: 500

And the connection is closed automatically.

I'm familiar with the following threads but none worked for me:

https://gist.github.com/jsdevtom/7045c03c021ce46b08cb3f41db0d76da#file-ingress-service-yaml

https://github.com/kubernetes/ingress-nginx/issues/3746

Any help will be much appreciated.

回答1:

Looks like the issue was on the Cloudflare side, we test the current solution against a different domain and it's working perfectly.

Few articles and answers that might help:

cloudflare-and-socket-io

Socket.io + nginx + cloudflare problems

Socket.io over https with cloudflare

来源：https://stackoverflow.com/questions/61730357/websocket-returns-500-on-client-and-101-on-server

标签

nginx

Kubernetes

websocket

kubernetes-ingress

nginx-ingress