gRPC call, channel, connection and HTTP/2 lifecycle

Question

I read the gRPC Core concepts, architecture and lifecycle, but it doesn\'t go into the depth I like to see. There is the RPC call, gRPC channel, gRPC connection (not described i

Answer 1

I was actually waiting for Eric to answer this as he is the expert in this!

I also have been playing with gRPC for a while now, I would like to add few things here for beginners. Anyone more experienced, please feel free to edit!

---

Channel is an abstraction over a long-lived connection! The client application will create a channel on start up. The channel can be reused/shared among multiple threads. It is thred safe. One channel is enough (for most of the use cases) for multiple threads and multiplexing concurrent requests. It is channel's responsibility to close / reconnect / keep the connection alive etc. We as the users do not have to worry about this in general. The client application can close the channel anytime it wants. Channel creation seems to be an expensive process. So we would not open/close for every RPC.

When you use gRPC loadbalancer/nameresolver for a domain name and the nameresolver resolves the domain with multiple ip addresses, a channel creates multiple subchannels where each subchannel is an abstraction over a connection to 1 server. So a channel can also represent multiple connections!!

Adding some points to note from Eric's comment.

adding the default load balancer still only creates (approximately) one connection if the name resolver returns multiple addresses, as the default is pick_first. But if you change the load balancer to round_robin or virtually any other policy, then yes, there will be multiple connections in a channel. Even if a name resolver returns one address, the load balancer is free to create multiple connections (e.g., for higher throughput), but that's not common today

An underlying connection can be closed any time for any reason. For ex: remote server is shutting down gracefully for a scheduled maintenance or a connection is idle for longer duration. In that case, the server could send GOAWAY signal to the client and client might disconnect and reconnect to some other server. or Server might crash due to OOM error. In this case channel will detect connection failure and will retry for new connection for some other server etc.

A channel can keep sending PING frame to the server to keep the connection alive. These are all configurable via channel builder.

With these information above, if we look at your questions,

what happens to the channel when a RPC throws an exception? Nothing happens to the channel. The unhandled exception on the server might the fail the RPC on the client side. But channel is still usable for any RPC calls.

What happens to the gRPC connection when the channel is closed? Channel is an abstraction over the connection. So it will be closed. (again there is no gRPC connection as such as Eric had mentioned. It would be a HTTP2 connection)

When is the channel closed? Any time you want. But normally when the application shuts down.

When is the gRPC connection closed? It is not our problem. Channel takes care of this.

Heart beats? Channel sends PING frames periodicaly to keep the connection alive.

What if the deadline is exceeded? It is something like timeout on the client side. When the deadline exceeds, the client might cancel the request. Once again nothing happens to the channel. (But it might trigger exception on the server side which I had noticed few times. (Received DATA frame for an unknown stream. https://github.com/grpc/grpc-java/issues/3548). It seems to have been fixed now).

Answer 2

The connection is not a gRPC concept. It is not part of the normal API and is an implementation detail. This should be seen as fairly normal, like HTTP libraries providing details about HTTP exchanges but not exposing connections.

It is best to view RPCs and connections as two mostly-separate systems.

The only real guarantee is that "connections are managed by channels," for varying definitions of "managed." You must shut down channels when no longer used if you want connections and other resources to be freed. Other details are either an implementation detail or an advanced API detail.

There is no "gRPC connection." A "gRPC connection" would just be a standard "HTTP/2 connection." Except that is even an implementation detail of the transport in many gRPC implementations. That allows having alternative "connection" types like "inprocess" or QUIC (via Cronet, where there is not a classic "connection" at all).

It is the channel's job to hold all the connections and reconnect as necessary. It delegates part of that responsibility to load balancers and the load balancing APIs do have a concept of connections (subchannels). By not exposing connections to the application, load balancers have a lot of freedom to operate.

I'll note that gRPC C-core based implementations share connections across channels.

What happens to the channel when a RPC throws an exception?

The channel and connection is not impacted by a failed RPC. Note that connection-level failures typically cause RPCs to fail. But things like retries could allow the RPC to be re-sent on a new connection.

What happens to the gRPC connection when the channel is closed?

The connections are closed, eventually. Channel shutdown isn't instantaneous because existing RPCs can continue, and connection shutdown isn't instantaneous as well. But once all RPCs complete the connections are closed. Although C-core won't shut down a connection until no channels are using it.

When is the channel closed?

Only when the user closes it.

When is the gRPC connection closed?

Lots of times. The client may close it when no longer needed. For example, let's say the server IP address changes and the client need to connect to 1.1.1.2 instead of 1.1.1.1. A new connection will be created and new RPCs will go to the new IP address. The client may also close connections it thinks are dead (e.g., via keepalive timeouts).

Servers have a lot of say of when to close connections. They may close them simply because they are old, or because they have been idle, or because the server is overloaded. But those are simply use-cases; the server can shut down a connection at-will.

What if the deadline is exceeded?

Deadline only applies to RPCs and doesn't impact the channel or a connection.