The second in the series of Simplifying Sparse Deep Learning with Universal Sparse Tensor in nvmath-python In a previous post, NVIDIA introduced the Universal Sparse Tensor (UST), enabling developers to decouple a tensor’s sparsity from its memory layout for greater flexibility and performance. We’re excited to announce the integration of the UST into nvmath-python v0.9.0 to accelerate sparse scientific and deep learning applications. Read the follow-on post: https://t.co/sARnNMbz1r #PyTorch #OpenSourceAI #AI #Inference #Innovation

🚀 Day-0 MTP support for Gemma4 now available at vLLM with ready-to-use docker image! ⚡️Enjoy up to 3x faster decoding performance to supercharge your development with zero quality degradation! Check out the full vLLM recipes for Gemma 4 model series👇 https://t.co/IrCaaa6SIo https://t.co/eFcAZRogLF

FINAL CALL for savings! ⏳ Today is the absolute last day for early bird rates for #KubeCon + #CloudNativeCon + #OpenInfraSummit + #PyTorchCon China (8-9 Sept). Join us in Shanghai and save ¥1,270 RMB if you register by 23:59 CST tonight: https://t.co/9x7dLq0gVE https://t.co/OmOVEzgTxR

Introducing TokenSpeed, a speed-of-light LLM inference engine. > TensorRT LLM level performance > vLLM level usability > Built by a lean and mission-driven team in two months > MIT license, open-source https://t.co/MJzhCEg7m8 https://t.co/anhoETwwS9 https://t.co/BWn4Me62x7

+1 more

🚀 Excited to be the exclusive day-0 launch partner for @lightseekorg's Tokenspeed project! We've integrated Tokenspeed's MLA library, optimized specifically for agentic workloads with long context and multi-turn, purpose-built for Kimi 2.5/2.6 and DeepSeek R1 on NVIDIA Blackwell hardware! Try it out today with our preview image - nightly support coming soon!

From #Kernel Engineering to Responsible #AI: We want your voice at #PyTorchCon North America in San Jose (Oct 20-21). 🔥 Submit your talk by June 7 & join the world's leading AI innovators. #CallForProposal submissions: https://t.co/hLlKK7WxLD https://t.co/fLsCa4jaJX

AI-Powered Workflows to Build Fault-Tolerant Quantum Systems The fundamental challenge in quantum computing—qubits are inherently noisy. The best quantum processors make an error roughly once in every thousand operations. To become useful accelerators for scientific and enterprise problems, error rates must drop to one in a trillion or better. AI is the most promising path to closing that gap at scale. NVIDIA Ising is the world’s first family of open AI models for building quantum processors, launching with two model domains: Ising Calibration and Ising Decoding. 🔗 Read the full post: https://t.co/0m20o5x8FU #PyTorch #OpenSourceAI #AI #Inference #Innovation

We’re excited to share that Unsloth has joined the PyTorch Ecosystem! Unsloth is an open-source project that makes training & running models faster, more accurate with less compute. We want AI to be accessible to all Blog: https://t.co/viXIukVHnU GitHub: https://t.co/aZWYAtakBP https://t.co/7DBBpeyIot

+1 more

Facing the computational chemistry trade-offs between accuracy and speed? Machine learning interatomic potentials (MLIPs) have emerged as a bridge, offering quantum accuracy at classical speeds. NVIDIA has introduced a collection of GPU-accelerated simulation building blocks that are designed to manage the data flow between accelerated chemistry and materials domain-specific kernels and deep learning models. NVIDIA ALCHEMI Toolkit extends beyond individual models and kernels to provide a modular, PyTorch-native structure for researchers and developers to compose custom simulation workflows. 🔗 Read the full post: https://t.co/jZ4LsGpva0 #PyTorch #OpenSourceAI #AI #Inference #Innovation

What if instead of building one giant AI, we evolved a coordinator to orchestrate a diverse team of specialized AIs? 🐟 Excited to share our new paper: “TRINITY: An Evolved LLM Coordinator”, published as a conference paper at #ICLR2026! Paper: https://t.co/Pr1TT15iqa In nature, complex problems are rarely solved by a single monolithic entity, but rather by the coordinated efforts of specialized individuals working together. Yet, modern AI development is heavily focused on endlessly scaling up single, massive monolithic models, yielding diminishing returns. While model merging offers a way to combine different skills, it is often impractical due to mismatched neural architectures and the closed-source nature of top-performing models. To address this, we took a macro-level approach: test-time model composition. We introduce TRINITY, a system that fuses the complementary strengths of diverse, state-of-the-art models without needing to modify their underlying weights. TRINITY processes queries over multiple turns. At each step, a lightweight coordinator assigns one of three distinct roles to an LLM from its available pool: 1/ Thinker: Devises high-level strategies and analyzes the current state. 2/ Worker: Executes concrete problem-solving steps. 3/ Verifier: Evaluates if the current solution is complete and correct. By dynamically assigning these roles, the coordinator effectively offloads complex reasoning and skill execution onto the external models. What makes TRINITY unique is its extreme efficiency. The coordinator relies on the hidden states of a compact language model and a small routing head. In total, it has fewer than 20K learnable parameters. Training this system presented a massive challenge. Traditional Reinforcement Learning (REINFORCE) failed because the gradients had a low signal-to-noise ratio due to binary rewards and weak parameter coupling. Imitation learning (Supervised Fine-Tuning) was ruled out because generating multi-turn labels is prohibitively expensive. Our solution? We turned to nature-inspired algorithms. We optimized the coordinator using a derivative-free evolutionary algorithm. We found that evolution is uniquely suited to optimize this tight, high-dimensional coordination problem where traditional gradient-based methods fail. The results are very promising. In our experiments, TRINITY consistently outperforms existing multi-agent methods and individual models across various benchmarks. At the time of publication, it set a new state-of-the-art record on LiveCodeBench, achieving an 86.2% pass@1 score. More importantly, it demonstrated incredible generalization. Without any retraining, TRINITY transferred zero-shot to four unseen tasks (AIME, BigCodeBench, MT-Bench, and GPQA). On average, the evolved coordinator surpassed every individual constituent model in its pool, including GPT-5, Gemini 2.5-Pro, and Claude-4-Sonnet (the top frontier models available at the time of our #ICLR2026 submission last year). This work is central to Sakana AI's vision. We believe the future of AI isn't just about scaling monolithic models, but engineering collaborative, diverse AI ecosystems that can adapt and combine their strengths. We invite the community to read the paper and explore these ideas! Paper: https://t.co/Pr1TT15iqa OpenReview: https://t.co/mbSP1aFCv7 This foundational research is part of the core engine powering our multi-agent product: Sakana Fugu 🐡👇

Scaling massive monolithic LLMs continues to yield incredible results. But to truly unlock their ceiling, the next frontier is test-time compute and dynamic orchestration. Nature solves complex problems through collaborative ecosystems. In our new #ICLR2026 paper, we evolved a small coordinator. Instead of competing with the monoliths, it orchestrates them. It learns to dynamically assign Thinker, Worker, and Verifier roles to a pool of frontier models—combining their strengths to hit SOTA on LiveCodeBench. This research is part of the engine powering our new product: Sakana Fugu https://t.co/ucZJke5ZaX 🐡

TRINITY: An Evolved LLM Coordinator https://t.co/5sSaKjUG8N https://t.co/pFXyONzNjK

Introducing our new work: “Learning to Orchestrate Agents in Natural Language with the Conductor” accepted at #ICLR2026 https://t.co/Wnh9ZACmLm What if we trained an AI not to solve problems directly, but to act as a manager that delegates tasks to a diverse team of other AIs? To solve complex tasks, humans rarely work alone; we form teams, delegate, and communicate. Yet, multi-agent AI systems currently rely heavily on rigid, human-designed workflows or simple routers that just pick a single model. We wanted an AI that could dynamically build its own team. We trained a 7B Conductor model using Reinforcement Learning to orchestrate a pool of frontier models (including GPT-5, Gemini, Claude, and open-source models available during the period leading up to ICLR 2026). Instead of executing code, the Conductor outputs a collaborative workflow in natural language. For any given question, the Conductor specifies: 1/ Which agent to call 2/ What specific subtask to give them (acting as an expert prompt engineer) 3/ What previous messages they can see in their context window Through pure end-to-end reward maximization, amazing behaviors emerged. The Conductor learned to adapt to task difficulty: it 1-shots simple factual questions, but autonomously spins up complex planner-executor-verifier pipelines for hard coding problems. The results are very promising: The 7B Conductor surpasses the performance of every individual worker model in its pool, setting new records on LiveCodeBench (83.9%) and GPQA-Diamond (87.5%) at the time of publication. It also significantly outperforms expensive multi-agent baselines like Mixture-of-Agents at a fraction of the cost. One of our favorite features: Recursive Test-Time Scaling! By allowing the Conductor to select itself as a worker, it reads its own team's prior output, realizes if it failed, and spins up a corrective workflow on the fly. This opens a new axis for scaling compute during inference. This research proves that language models can become elite meta-prompt engineers, dynamically harnessing collective intelligence. Alongside our TRINITY research which we announced a few days earlier, this foundational research powers our new multi-agent system: Sakana Fugu! (https://t.co/36Ud311KCP) 🐡 OpenReview: https://t.co/e5WqTleQNL (ICLR 2026)

For the past few years, humans have been doing “prompt engineering” to coax the best performance out of different LLMs. In this work, we explored what happens if we train an AI to do that job instead. By training a Conductor model with RL, we found that it naturally learns to write highly effective, custom instructions for a whole pool of other models. It essentially learns to ‘manage’ them in natural language. What surprised me most was how it dynamically adapts. For simple factual questions, it just queries one model. But for hard coding problems, it autonomously spins up a whole pipeline of planners, coders, and verifiers. Really excited to see where this paradigm of “AI managing AI” goes next, especially as we start moving from single-agent chain-of-thought to multi-agent “chain-of-command”. Link to our #ICLR2026 paper: https://t.co/EwbjjRPLUb Along with our TRINITY paper which we announced earlier, this work also powers our new multi-agent system: Sakana Fugu (https://t.co/2m8VRdOYqG) 🐡

Learning to Orchestrate Agents in Natural Language with the Conductor Fugu Blog: https://t.co/7idk8sOsoU Paper: https://t.co/Xh7r5XAbkN 🐡 https://t.co/X7dxaeNUhp

We’re excited to introduce KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speech-to-Speech Conversational AI, accepted at #ICASSP2026! 🐢 Blog https://t.co/arVz1TGpJJ Paper https://t.co/0EwpyRXeCs Can a speech AI think deeply without pausing to process? In real conversation, we don’t wait until we’ve fully worked out what we want to say—we start talking, and our thoughts catch up as the sentence unfolds. Fast speech-to-speech models achieve this, but their reasoning tends to stay shallow. Cascaded pipelines that route through a knowledgeable LLM are smarter, but the added latency breaks the flow—they fall back to "think, then speak." In our new paper, we propose a way to break this trade-off. We call it KAME (Turtle in Japanese). A speech-to-speech model handles the fast response loop and starts replying immediately. In parallel, a backend LLM runs asynchronously, generating response candidates that are continuously injected as "oracle" signals in real time. This shifts the AI paradigm from "think, then speak" to "speak while thinking." The backend LLM is completely swappable. You can plug in GPT-4.1, Claude Opus, or Gemini 2.5 Flash depending on the task without changing the frontend. In our experiments, Claude tended to score higher on reasoning, while GPT did better on humanities questions. Try the model yourself here: https://t.co/uDA0nvvjhS

Sakana AIは、SMBCグループと共同で「提案書自動生成アプリケーション」を開発。三井住友銀行にて実務への適用を開始します。 https://t.co/LW9DPEf8Tu 複数の「AIエージェント」が自律的に連携し、情報収集から仮説構築、提案構成までを支援。日本を支える基幹産業のDXを、独自の技術で強力に 後押しします。 2025年のパートナーシップ締結以来、検討を重ねてきた成果がいよいよ「実装フェーズ第一号」として形になりました。Sakana AIは、今後も最先端技術の社会実装を次々と実現していきます。

GWの富山。 新緑の先に、まだ雪の立山連峰。 https://t.co/aa1WwoIOmn

GWの富山。 新緑の先に、まだ雪の立山連峰。 https://t.co/aa1WwoIOmn

If Japan built Github https://t.co/JCLnYqqmgK

If GitHub were built in: Japan 🇯🇵 China 🇨🇳 North Korea 🇰🇵 The EU 🇪🇺 https://t.co/LNMt3QxpBQ

+2 more

@SakanaAILabs Sakana Fugu: A Multi-Agent Orchestration System as a Foundation Model https://t.co/mtCq1eKpzk

How do we make LLMs faster and lighter? Don’t force the GPU to adapt to sparsity. Reshape the sparsity to fit the GPU! ⚡️ Excited to share our new #ICML2026 paper in collaboration with @NVIDIA: "Sparser, Faster, Lighter Transformer Language Models". This work introduces new open-source GPU kernels and data formats for faster inference and training of sparse transformer language models: Paper: https://t.co/3Avj8N8iYO Blog: https://t.co/SqFkkKvkbd Code: https://t.co/PHSzMq8pg0 While LLMs are undoubtedly powerful, they are increasingly expensive to train and deploy, with a large part of this cost coming from their feedforward layers. Yet, an interesting phenomenon occurs inside these layers: For any given token, only a small fraction of the hidden activations actually matter. The rest approximate zero, wasting computation. With ReLU and very mild L1 regularization, this sparsity can exceed 95% with little to no impact on downstream performance. So, can we leverage this sparsity to make LLMs faster? The challenge is hardware. Modern GPUs are optimized for dense matrix multiplications. Traditional sparse formats introduce irregular memory access and overheads that cancel out their theoretical savings for GEMM operations. Our contribution is twofold: 1/ We introduce TwELL (Tile-wise ELLPACK), a new sparse packing format designed to integrate directly in the same optimized tiled matmul kernels without disrupting execution. 2/ We develop custom CUDA kernels that fuse multiple sparse matmuls to maximize throughput and compress TwELL to a hybrid representation that minimizes activation sizes. We used our kernels to train and benchmark sparse LLMs at billion-parameter scales, demonstrating >20% speedups and even higher savings in peak memory and energy. This work will be presented at #ICML2026. Please check out our blog and technical paper for a deep dive!

The human brain🧠 is incredibly efficient because it only activates the specific neurons needed for a thought. Modern LLMs naturally try to do this too (> 95% of neurons in feedforward layers stay silent for any given word), but our hardware punishes them for it. One of the most frustrating paradoxes in deep learning: making a model do less math often makes it run slower. Why? Because unstructured sparsity introduces irregular memory access, and GPUs are built for predictable, dense blocks of math. We teamed up with @NVIDIA to try to fix this hardware mismatch. Instead of forcing the GPU to adapt to the sparsity, we built a "Hybrid" format that reshapes the sparsity to fit the GPU. Our sparsity format (TwELL) dynamically routes the 99% of highly sparse tokens through a fast path, and uses a dense backup matrix as a safety valve for the rare, heavy tokens. Through TwELL and a new set of custom CUDA kernels for both LLM inference and training, we translated theoretical sparsity into actual wall-clock speedups: >20% faster training and inference on H100 GPUs, while also cutting energy consumption and memory requirements. Paper: https://t.co/rqIY9SYBDe Blog: https://t.co/oRjNbpJKha Code: https://t.co/FAFaJwpxAJ ⚡️

If you want to look under the hood at the actual custom CUDA kernels and see exactly how we implemented the TwELL format for H100 GPUs, we’ve released the reference code. GitHub: https://t.co/FAFaJwpxAJ Blog: https://t.co/oRjNbpJKha 🐟

Reproducing all of Schmidhuber’s papers (1990-2025) using an AI coding assistant. Cool project by @yaroslavvb! It even reproduced the “World Models” paper by me and @SchmidhuberAI with a toy env, with a full VAE + RNN world model implementation. Project: https://t.co/sgQG5umNEm https://t.co/iKMFN7ti9z

Machine learning engineering (MLE) is the new agentic frontier. I'll be sharing our work on scaling RL for MLE agents at #ICLR2026: 1) RL of a small model outperforms a frontier model https://t.co/vfk1jc76t5 2) MLE-Smith: scale-up MLE tasks automatically https://t.co/Mc5vyY2fhg https://t.co/fpCcvyBvAU

🚀SonicMoE🚀now runs at peak throughput on NVIDIA Blackwell GPUs 😃 54% & 35% higher fwd/bwd TFLOPS than the DeepGEMM baseline and 21% higher fwd TFLOPS than the triton official example. SonicMoE still maintains its minimum activation memory footprint: the same as a dense model with equal activated parameters and independent of expert granularity. We wrote a blogpost on how we leveraged Blackwell features and the software abstraction on QuACK: Work with @MayankMish98, @XinleC295, @istoica05, @tri_dao

Introducing @NeoCognition, the agent lab for specialized intelligence. Everyone needs experts, but human expertise does not scale. Backed by $40M seed funding, we build self-learning agents that specialize across domains to make expertise abundant. https://t.co/Oh4VawqlwP

Self-play led to superhuman Go performance, why hasn’t it for LLMs? In practice, long run self-play plateaus like RL. We study why this happens, and build a self-play algorithm that scales better. It solves as many problems with a 7B model as the pass@4 of a model 100x bigger. https://t.co/zFH6vICnul

New modded-NanoGPT optimization benchmark result: @wen_kaiyue has improved upon both the Muon and AdamW baselines, by replacing their weight decay with hyperball optimization. The new record is 3325 steps. https://t.co/8hitPuZmU7

Playing with an optimizer speedrun is something that never gets old. Built on top of https://t.co/n7gbPOCTPT and Claude should take all the credit for hypertl tuning.