Google Brings PyTorch to Its AI Chips, and Meta Says It Is Already in Production
Google released native PyTorch backends for the vLLM and SGLang inference engines on its TPU chips on Oct. 20, 2026, and a Meta engineering lead quoted in the announcement says one of the two is already handling production traffic at Meta.
Both engines are serving software: the layer that takes a trained model and answers a user's requests with it. PyTorch is the framework most AI models are written in. Google's launch post says the two engines already ran on its TPUs through code written in JAX, Google's own framework, while developers wanted to customize and debug their serving stack in PyTorch. The new backends sit on TorchTPU, which makes a TPU behave like an ordinary PyTorch device, and the stated goal is to let the PyTorch community run models on TPUs with tools those developers already use.
"SGLang-TorchTPU is running in production at Meta today, and getting there was a close three-way collaboration between Meta, Google, and the SGLang team to make the stack production-ready," said Zongwei Zhou, PyTorch-TPU lead at Meta, in the announcement. Google describes both engines as running at scale in production with preview customers, whom it does not name.
The code is open source, and Google has published GitHub repositories for TorchTPU and for both engines. The vLLM backend was co-developed with Inferact and the SGLang backend with RadixArk, companies founded by the creators of the two projects.
The performance figures released alongside the engines are Google's own submissions to the SemiAnalysis InferenceX benchmark project, and no independent evaluation of them has been published. Google puts its lowest cost at 2.01 cents per million tokens and reports more than 700 tokens per second to a single user on Kimi K3, a 2.8-trillion-parameter model, running on its Ironwood TPU.
Google says its earlier JAX versions of both engines will continue to be maintained, and that the vLLM project is opening a pool of free TPU capacity for developers who contribute to the engine.
