Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets

Google, Red Hat, and IBM have released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference featuring cache-aware routing for TPU and…