AiAnyTool - AI Tools Directory LogoAiAnyTool - AI Tools Directory Logo
AIAnyToolAI Tools Directory
Full ListAI CategoriesAI News Today
Loading theme toggle
Submit an AISign up
AiAnyTool - AI Tools Directory LogoAiAnyTool - AI Tools Directory Logo
AiAnyTool

Artificial intelligence for everyone. Discover, compare, and leverage over 19,000 verified AI tools.

Updated daily with manual verification

Resources

  • AI Deals & Discounts
  • AI News Today
  • Tutorials & Guides
  • MCP Servers Directory

Directory

  • Full AI List
  • AI Categories
  • Top 10 Best AI
  • New AI Tools
  • Submit an AI Tool →

About & Legal

  • About AiAnyTool
  • Contact Support
  • Privacy Policy
  • Terms of Service

© 2026 AiAnyTool.com — All rights reserved.

The definitive artificial intelligence directory and search engine.

HomeAI NewsOpen Source

Hugging Face ❤️ TorchTPU: TPUs, Natively

Hugging FaceOpen SourceOctober 20, 2026(Oct 20)5 min read
Share:
Hugging Face ❤️ TorchTPU: TPUs, Natively

Full Coverage Details

Hugging Face Blog

Back to Articles a]:hidden"> Hugging Face ❤️ TorchTPU: TPUs, Natively Published October 20, 2026 Update on GitHub Upvote 54 +48 Alvaro Moran tengomucho Follow Jingya Huang Jingya Follow Simon Pagezy pagezyhf Follow Jeff Boudier jeffboudier Follow Régis Pierrard regisss Follow :last-child]:mb-0"> Wait, what's a TPU? 🤔 The old way Enter TorchTPU 🚀 Your favourite libraries, now on TPU Transformers Images? Videos? Diffusers paints them on TPUs 🎨 Kernels We test it so you don't have to 🧪 Get started If you ever tried to run PyTorch on a TPU, you probably ported your code to torch_xla, or you ended up having to learn JAX. With TorchTPU, a native PyTorch backend, TPU is now just another device. It opens up native support of TPU in the Hugging Face ecosystem. Transformers, Diffusers and Kernels run natively on TPU. Pick a model from the Hub, send it to "tpu", and you're done.

Wait, what's a TPU? 🤔 TPU stands for Tensor Processing Unit. TPUs are Google's own AI accelerators. They are chips built for one job: the matrix math that neural networks spend most of their time on. Google has been designing them for about ten years, and uses them to train and serve its own models, including Gemini.

TPUs are available on Google Cloud, from a single chip up to pods of thousands of chips. They shine when you need a lot of compute, for training or for serving at scale.

The catch has always been software. The TPU stack grew up around TensorFlow and JAX, and PyTorch users had to work harder to get in.

The old way Until now, running PyTorch on TPUs meant going through PyTorch/XLA (torch_xla). It worked, but it had its quirks. Tensors were lazy (the tensors, not you), so your code didn't always run when you thought it did. Small changes could trigger graph recompilations, and debugging was hard to reason about.

To make life easier, we built Optimum TPU. It gave Hugging Face models inference and training support on TPUs. Now that TorchTPU brings native support, Optimum TPU is not required anymore to run Hugging Face models with Transformers and Diffusers.

Enter TorchTPU 🚀 TorchTPU adds TPU as a new backend to PyTorch. You write regular PyTorch code. It comes with Pallas kernels tuned for TPUs, support for distributed training, and a torch.compile integration for when you want more speed. Installing it on a TPU VM takes one command:

pip install torch-tpu --extra-index-url https://download.pytorch.org/whl/cpu The extra index gives you the CPU build of torch. Without it, pip pulls the much bigger CUDA build, which you don't need on a TPU.

Your favourite libraries, now on TPU Transformers Transformers v5 was built around one idea: be the source of truth for model definitions, so they can run anywhere. "Anywhere" now includes TPUs. To load a model on a TPU, pass device_map="tpu". That's it.

from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Qwen/Qwen3.5-9B" model = AutoModelForCausalLM.from_pretrained( model_id, device_map="tpu", # will load model directly on the TPU ) tokenizer = AutoTokenizer.from_pretrained(model_id) inputs = tokenizer("The future of AI is", return_tensors="pt").to("tpu") out = model.generate(**inputs, max_new_tokens=64) print(tokenizer.decode(out[0], skip_special_tokens=True)) The model runs in eager mode by default. You can also use torch.compile for better performance, or run it in a distributed setup.

Images? Videos? Diffusers paints them on TPUs 🎨 TPU is now a first-class device in Diffusers. For prototyping or debugging, eager mode only needs pipe.to("tpu"). For deployment, compiled mode adds one compile call. Your pipelines and code stay the same, and there's no TPU fork to maintain.

import torch from diffusers import StableDiffusion3Pipeline pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3.5-medium", dtype=torch.bfloat16 ) pipe.to("tpu") # eager mode: that's the whole change # optional: compiled mode pipe.transformer.compile(backend="tpu", fullgraph=True, dynamic=False) image = pipe("a corgi astronaut planting a flag on a pizza moon").images[0] The first call compiles, and every call after that with the same image size reuses the graph.

Models too big for one chip? Pass a TensorParallelConfig to from_pretrained and the transformer is sharded across chips as it loads, so the full model never has to fit in one place. Many popular pipelines support it out of the box, and the ones that don't are just a tp_plan away. The Diffusers TPU guide has the full recipe.

TP=4 (each model split across four chips) · bf16 · batch 1

Kernels The Kernels Hub has ready-to-use TPU kernels. Load one with the Kernels library and give your model a speed boost.

H

Hugging Face

Original Reporting Source

Read Full Article at Source

AI Tools Mentioned in this Story

Explore features, ratings, and alternatives from our 19,000+ AI tools catalog.

Explore Directory

AssistLoop

Free plan with 150 message credits and basic features, Basic at $25/month for 3,000 credits and lead collection, Pro at $109/month for 15,000 credits and advanced actions, Business with custom pricing for 50,000+ credits and unlimited agents.

AssistLoop's AI chatbots slash response times by 70% and boost satisfaction, scaling support for e-commerce and SaaS without extra hires.

AlternativesView Tool

GetResponse AI Website Builder

Free + from $10.23/mo

Design professional websites effortlessly with AI.

AlternativesView Tool

HyperTxt

Lite plan at $15/month with 20 credits, Pro at $39/month (most popular) with 100 credits and priority support, Pro+ at $125/month with 500 credits and advanced features, free trial available on all paid plans.

Hypertxt.ai is an AI-powered SEO content writing tool that helps create high-quality, optimized content for improved search engine rankings. It offers various features to streamline the content creation process, from keyword research to content optimization and plagiarism checks.

AlternativesView Tool
← Back to AI News RadarExplore Trending Stories
Trending AI Stories
View All
#1

Signal blocks screenshots on iOS 27 and Android to stop AI agents from reading private chats

The Decoder • 3h ago
#2

Google's Gemini can now run Mac apps and finish desktop chores in the background while you keep working

The Decoder • 4h ago
#3

Anthropic mathematician claims to have solved prestigious Millennium Prize problem with Claude

The Verge AI • 6h ago
#4

Apple is hiring engineers to build its own AI agent

The Verge AI • 8h ago
#5

Apple joins the race for AI agents that control your device while raising new barriers for competitors

The Decoder • 8h ago

More in Open Source

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

2d ago
Why your fine-tuning job takes longer than the FLOPs say

Why your fine-tuning job takes longer than the FLOPs say

2d ago
180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

2d ago
Hugging Face and OpenRouter join forces to make open-weight models easy to run

Hugging Face and OpenRouter join forces to make open-weight models easy to run

4d ago