@PyTorch
vLLM and PyTorch worked together to fix a long-standing aarch64 install headache ā as of PyTorch 2.11.0, pip install torch on GB200 / GB300 / GH200 just works. š What changed: PyTorch 2.11.0 now publishes CUDA-enabled aarch64 wheels to the default PyPI index. No more custom --index-url flags. No more transitive dependencies silently swapping your GPU build for the CPU wheel. New users on Grace Hopper and Grace Blackwell systems can follow the standard install instructions and have vLLM work the first time. In our latest blog, @KaichaoYou (co-founder @inferact, Lead Maintainer @vllm_project) shares the full story: š A 2024 hackathon bug bringing up vLLM on GH200 š§ vLLM's in-tree workarounds (use_existing_torch.py and [tool.uv] build-isolation passthrough) š¤ From GitHub issue to PyTorch Foundation TAC discussion š The fix landing in PyTorch 2.11.0, driven by NVIDIA and PyTorch core. A great example of cross-project collaboration under the PyTorch Foundation umbrella ā and a reminder that boring infrastructure wins compound. Read the full story: https://t.co/JGnJ1X7sxl āļø : Piotr Bialecki (@nvidia) ā @ptrblck_de, Alban Desmaison (@Meta), Andrey Talman (@Meta), Nikita Shulga (@Meta)