Kubernetes: how to access service if nodePort is random?

问题

I'm new to K8s and am currently using Minikube to play around with the platform. How do I configure a public (i.e. outside the cluster) port for the service? I followed the nginx example, and K8s service tutorials. In my case, I created the service like so:

kubectl expose deployment/mysrv --type=NodePort --port=1234

The service's port is 1234 for anyone trying to access it from INSIDE the cluster. The minikube tutorials say I need to access the service directly through it's random nodePort, which works for manual testing purposes:

kubectl describe service mysrv | grep NodePort
...
NodePort:                 <unset>  32387/TCP
# curl "http://`minikube ip`:32387/"

But I don't understand how, in a real cluster, the service could have a fixed world-accessible port. The nginx examples describe something about using the LoadBalancer service kind, but they don't even specify ports there...

Any ideas how to fix the external port for the entire service?

回答1:

The minikube tutorials say I need to access the service directly through it's random nodePort, which works for manual testing purposes:

When you create service object of type NodePort with a $ kubectl expose command you cannot choose your NodePort port. To choose a NodePort port you will need to create a YAML definition of it.

You can manually specify the port in service object of type Nodeport with below example:

apiVersion: v1
kind: Service
metadata:
  name: example-nodeport
spec:
  type: NodePort
  selector:
    app: hello # selector for deployment
  ports:
  - name: example-port
    protocol: TCP
    port: 1234 # CLUSTERIP PORT
    targetPort: 50001 # POD PORT WHICH APPLICATION IS RUNNING ON 
    nodePort: 32222 # HERE!

You can apply above YAML definition by invoking command: $ kubectl apply -f FILE_NAME.yaml

Above service object will be created only if nodePort port is available to use.

But I don't understand how, in a real cluster, the service could not have a fixed world-accessible port.

In clusters managed by cloud providers (for example GKE) you can use a service object of type LoadBalancer which will have a fixed external IP and fixed port.

Clusters that have nodes with public IP's can use service object of type NodePort to direct traffic into the cluster.

In minikube environment you can use a service object of type LoadBalancer but it will have some caveats described in last paragraph.

A little bit of explanation:

NodePort

Nodeport is exposing the service on each node IP at a static port. It allows external traffic to enter with the NodePort port. This port will be automatically assigned from range of 30000 to 32767.

You can change the default NodePort port range by following this manual.

You can check what is exactly happening when creating a service object of type NodePort by looking on this answer.

Imagine that:

• Your nodes have IP's:
  • 192.168.0.100
  • 192.168.0.101
  • 192.168.0.102
• Your pods respond on port 50001 with hello and they have IP's:
  • 10.244.1.10
  • 10.244.1.11
  • 10.244.1.12
• Your Services are:
  • NodePort (port 32222) with:
    • ClusterIP:
      • IP: 10.96.0.100
      • port:7654
      • targetPort:50001

A word about targetPort. It's a definition for port on the pod that is for example a web server.

According to above example you will get hello response with:

• NodeIP:NodePort (all the pods could respond with hello):
  • 192.168.0.100:32222
  • 192.168.0.101:32222
  • 192.168.0.102:32222
• ClusterIP:port (all the pods could respond with hello):
  • 10.0.96.100:7654
• PodIP:targetPort (only the pod that request is sent to can respond with hello)
  • 10.244.1.10:50001
  • 10.244.1.11:50001
  • 10.244.1.12:50001

You can check access with curl command as below:

$ curl http://NODE_IP:NODEPORT

---

In the example you mentioned:

$ kubectl expose deployment/mysrv --type=NodePort --port=1234

What will happen:

• It will assign a random port from range of 30000 to 32767 on your minikube instance directing traffic entering this port to pods.
• Additionally it will create a ClusterIP with port of 1234

In the example above there was no parameter targetPort. If targetPort is not provided it will be the same as port in the command.

Traffic entering a NodePort will be routed directly to pods and will not go to the ClusterIP.

From the minikube perspective a NodePort will be a port on your minikube instance. It's IP address will be dependent on the hypervisor used. Exposing it outside your local machine will be heavily dependent on operating system.

---

LoadBalancer

There is a difference between a service object of type LoadBalancer(1) and an external LoadBalancer(2):

• Service object of type LoadBalancer(1) allows to expose a service externally using a cloud provider’s LoadBalancer(2). It's a service within Kubernetes environment that through service controller can schedule a creation of external LoadBalancer(2).
• External LoadBalancer(2) is a load balancer provided by cloud provider. It will operate at Layer 4.

Example definition of service of type LoadBalancer(1):

apiVersion: v1
kind: Service
metadata:
  name: example-loadbalancer
spec:
  type: LoadBalancer
  selector:
    app: hello
  ports:
    - port: 1234 # LOADBALANCER PORT 
      targetPort: 50001  # POD PORT WHICH APPLICATION IS RUNNING ON 
      nodePort: 32222 # PORT ON THE NODE 

Applying above YAML will create a service of type LoadBalancer(1)

Take a specific look at:

  ports:
    - port: 1234 # LOADBALANCER PORT 

This definition will simultaneously:

• specify external LoadBalancer(2) port as 1234
• specify ClusterIP port as 1234

Imagine that:

• Your external LoadBalancer(2) have:
  • ExternalIP: 34.88.255.5
  • port:7654
• Your nodes have IP's:
  • 192.168.0.100
  • 192.168.0.101
  • 192.168.0.102
• Your pods respond on port 50001 with hello and they have IP's:
  • 10.244.1.10
  • 10.244.1.11
  • 10.244.1.12
• Your Services are:
  • NodePort (port 32222) with:
    • ClusterIP:
      • IP: 10.96.0.100
      • port:7654
      • targetPort:50001

According to above example you will get hello response with:

• ExternalIP:port (all the pods could respond with hello):
  • 34.88.255.5:7654
• NodeIP:NodePort (all the pods could respond with hello):
  • 192.168.0.100:32222
  • 192.168.0.101:32222
  • 192.168.0.102:32222
• ClusterIP:port (all the pods could respond with hello):
  • 10.0.96.100:7654
• PodIP:targetPort (only the pod that request is sent to can respond with hello)
  • 10.244.1.10:50001
  • 10.244.1.11:50001
  • 10.244.1.12:50001

ExternalIP can be checked with command: $ kubectl get services

Flow of the traffic: Client -> LoadBalancer:port(2) -> NodeIP:NodePort -> Pod:targetPort

Minikube: LoadBalancer

Note: This feature is only available for cloud providers or environments which support external load balancers.

-- Kubernetes.io: Create external LoadBalancer

On cloud providers that support load balancers, an external IP address would be provisioned to access the Service. On Minikube, the LoadBalancer type makes the Service accessible through the minikube service command.

-- Kubernetes.io: Hello minikube

Minikube can create service object of type LoadBalancer(1) but it will not create an external LoadBalancer(2).

The ExternalIP in command $ kubectl get services will have pending status.

To address that there is no external LoadBalancer(2) you can invoke $ minikube tunnel which will create a route from host to minikube environment to access the CIDR of ClusterIP directly.

回答2:

According to questions:

• But I don't understand how, in a real cluster, the service could have a fixed world-accessible port.
• When accessing the service from outside the entire cluster without minikube (on GKE for example), how is the nodePort even relevant?

In cloud provided Kubernetes clusters LoadBalancer type of service will have a external IP assigned by cloud provider. The desired service will be accessible by provided external IP. LoadBalancer will also create a NodePort type of service automatically to route the traffic internally. NodePort ports will not be accessible for direct connection with Node IP address with external traffic.

Created LoadBalancer will allow you to specify a port on which it willl respond to traffic.

Take a look at below reference with a link:

Declaring a service of type LoadBalancer exposes it externally using a cloud provider’s load balancer. The cloud provider will provision a load balancer for the Service, and map it to its automatically assigned NodePort.

--- Ovh.com: getting-external-traffic-into-kubernetes

来源：https://stackoverflow.com/questions/60017002/kubernetes-how-to-access-service-if-nodeport-is-random