@vllm_project
vLLM π€ TPU π Native PyTorch serving on TPUs is here! Built with @Google and @inferact, vLLM-TorchTPU brings familiar PyTorch tools to TPU inference, with prefix caching, speculative decoding, and disaggregated serving. π οΈ Excited to keep building with the community!