The latest Data Phoenix news tagged llm-d.
llm-d 1.0 brings stable routing APIs, tenant quotas, session-aware scheduling and upgrade controls to Kubernetes-based AI inference.
IBM Research and Red Hat say llm-d combined cache-aware routing, distributed KV caching and split inference stages to serve GLM-5.2 on an existing…