Load Balancing and Reliability for gRPC on Kubernetes

Kubernetes relies on layer-4 routing, which distributes traffic on a per-connection basis rather than per-request. Because gRPC utilizes long-lived HTTP/2 connections to maximize efficiency, Kubernetes sends all subsequent requests down the exact same initial pipeline instead of spreading them across available pods.

This inherent architectural clash leads to severe traffic imbalances and underutilized replicas, requiring an application-layer proxy to achieve true load balancing.