I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)! The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵 https://t.co/PKQdSpWdxP

MacVoice Full voice loop on a MacBook, running on a local LLM - Just talk over the agent to interrupt it. First spoken word of the reply arrives **~0.7 s after you stop talking**. Fits in **1.4 GB**. No PyTorch. Writeup + repo 👇 https://t.co/nfKHKeawK6 https://t.co/UIa6WvXcM2 🔊 S…

In Axolotl, we’ve addressed this with merge-aware LoRA training: we merge the LoRA into the base weights and cast to BF16 inside the forward pass, so the model trains on exactly the weights vLLM serves. Gradients pass straight through the rounding to the LoRA weights, so they kee…

If you're using LoRA for RL, it's most likely best to deploy your adapter unmerged. Depending on your learning rate, how many effective train steps, etc, you might be losing much of your training updates to rounding to BF16 when you merge the adapter to the base model. https://t.…

llama.cpp b11938 on M2 Pro 32 GiB (using ~3 GiB with the full 128K context) at ~110 t/s with c1 running with Pi! llama serve --hf-repo LiquidAI/LFM3-VL-2B-GGUF:Q4_K_M --host 0.0.0.0 --port 8080 -fa on -np 1 --top-p 1.0 --min-p 0.0 https://t.co/enITzfYaPQ

Super interesting result: H-Net learns to dynamically tokenize variable names in coding data as separate tokens, and it scales much better on code compared to BPE

We wrote a 37-page technical report on OCRia-1, our SOTA model for document parsing. ✅ It contains comprehensive evals against most other major open-weight OCR models (e.g. Infinity-Parser, LightOnOCR 3, Chandra 2, dots.mocr) on multiple OCR benchmarks: ParseBench, olmOCR-bench, …

TorchTitan has released major performance and architecture updates in native PyTorch, adding full 5D parallelism support for DeepSeek-V4, Qwen3.5, Kimi K3, and Muse Spark Open alongside the new TitanRL framework for unified post-training and agentic workloads. Scaling efficiency …

vLLM 🤝 TPU 🚀 Native PyTorch serving on TPUs is here! Built with @Google and @inferact, vLLM-TorchTPU brings familiar PyTorch tools to TPU inference, with prefix caching, speculative decoding, and disaggregated serving. 🛠️ Excited to keep building with the community!

TorchTPU enables seamless out-of-the-box model execution and TPU migration while delivering industry-competitive inference and training performance through eager execution optimizations, torch.compile integration, & native hardware acceleration. Learn more on the PyTorch blog: ht…

@googlecloud @Meta Read our latest blog introducing TorchTPU to learn more: https://t.co/WnaK5hLFBd

Bill Jia (@GoogleCloud) and Zongwei Zhou (@Meta) introduce TorchTPU, a native PyTorch backend for Google TPUs, and agentic developer workflows at #PyTorchCon 2026. TorchTPU is an open source collaboration between Google and Meta, developed in close partnership with PyTorch core m…

Today we’re releasing Muse Spark Open, the most capable US open-weight model to date. Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning. Key Model Specs: → 816B total / 42B active parameters → 1M token conte…

TPUs have different bottlenecks from GPUs: few cores so you don’t need yo think about parallelism as much, large VMEM so easier to fuse kernels, but vector units have lower throughput so anything non matmul require much more care

https://t.co/58tzGHHYIN No vector database. No ANN library. No Pinecone. The whole search index is a 22 MB `.npz` on disk + a single `https://t.co/sBTu66gHDF` call on GPU. 7,000 vectors × 768d. Direct cosine. **4.4 ms median retrieval.** If you've been told you "need" a vector DB…

Releasing FLAT: Flash Linear Attention on TPUs 🚀 Pallas training kernels for Linear Attention, Mamba-2 and GLA. On one TPU v6e, forward + backward is up to 3x faster than FLA on an H100 for GLA, and 1.5x faster for Mamba-2. 🧵 https://t.co/are10bcub8

Great paper from NVIDIA sharing an open midtraining recipe for agentic coding. NVIDIA kept SFT identical and added one midtraining stage. SWE-bench Multilingual went from 55.8 to 62.8 on Nemotron 3.5 Lightning. The stage sits between pretraining and post-training. It trains at 25…

We just open-sourced another pair of state-of-the-art Decision Models: pplx-decider-flash-v1.2: 4B params, 1 cent per million tokens (top 4B model on HF Decision Index) pplx-decider-v1.2: 27B params, 2 cents per million tokens (#2 on HF Decision Index overall - for now, will impr…

@vllm_project Independent repro of the Qwen3-TTS path on a stock RTX 5090: 47 ms median time to first audio through vLLM-Omni, vs ~2.3 s through plain transformers. Pins + gotchas (cu130, sm_120 nvcc, Triton) here: https://t.co/bf78sauiA6

The largest agentic LLM Inference dataset has arrived! 206B tokens · 12,002 sessions · 1,186,582 LLM requests · 1,213,347 tool calls The traces include tool names, arguments, durations, and more. Explore the dataset on @huggingface https://t.co/12hrLuupVF Paper coming soon! Stay …

I built an operating system where the AI agent isn’t another app. It’s part of the OS. Today, Herald OS is open source and you can install it yourself 🚀 It’s built around Hermes Agent by Nous Research. Talk or type to Hermes and it can work across your entire computer: • Open and…

We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public…

paper: https://t.co/XvQqe6Kynr code: https://t.co/f28BWhEGEQ

When we started the run, our MFU was 21%. Over the course of training, we kept on iterating on better kernels (ragged EP, FA4 attention kernels) and improved the MFU to 27% and reduced the MoE drop rate to near zero. https://t.co/R1X29POGZx

Reasoning from scratch, round number 6! An introduction (and implementation) of Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). 00:00 Introduction 01:54 What makes a reasoning model different? 04:25 Reasoning traces and model c…

I think I just trained NanoGPT in 9.65 seconds!? Yesterday, @hermanbrunborg used a connected longest exact match model, trained concurrently on CPU, to get the nanoGPT record from 39.9 seconds to 21.5 seconds, in an impressive open PR. By adapting their work, and generalizing the…

New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with inc…

GPT-6 Sol from @OpenAI on ARC-AGI (Verified): - ARC-AGI-3: 4.6%, $5.6K (standard harness), 23.0%, $8.7K (provider adapter harness) - ARC-AGI-2: 89.6%, $0.44/task - ARC-AGI-1: 95.5%, $0.14/task Sol reaches 23.0% on ARC-AGI-3, vs Luna's 0.59% and Astra's 99.9%. https://t.co/Kjoz0GE…

only allowing Qwen to output the 250 most common tokens in Wikipedia, plus anything rarer than the 14,000th https://t.co/cV4bGgOQ3I

I evaluated Sonnet 5.5 on induction. It does very well! Coming 3rd after Astra and Opus 5.5 (a bit distant third) and a little above GPT-6 Sol. Its total token usage was heavy, with 57.6M output tokens over four rounds across the problems to get to a high evaluable rate. I also c…

Releasing our runtime dynamic compression framework that achieves 1.5-2.0 bit compression at high quality. This is integrated into the bitsandbytes2 library, which starts as a private beta today. Paper: https://t.co/f4L4Nm8JwM Private beta signup: https://t.co/q8PPAulmq3 https://…

Our 2nd release in our open-source week. CliffCompaction lets you run sessions for millions of tokens and save 50% on token costs. It integrates easily with Claude Code and Codex. We have been using it for months, and it has better vibes than Codex/Claude Code compaction.

Full walkthrough — enrollment (two embeddings, why), pipeline order (gate → TSE → Whisper), why guard-word is deterministic string matching (not a wake-word model), where addressee detection has to go, and the note mode that fell out of building modes right: https://t.co/ak1B2ocN…

CliffCompaction, our autocompaction tool for coding agents, is out! Sessions run for millions of tokens, agents stay on track, and costs drop by up to 50%. Paper: https://t.co/OdPp6Mt1Gh Code: https://t.co/39Mg0ZJZ9C PyPI: https://t.co/yOSIP3KCUH Blog: https://t.co/cZdN9ZbURf htt…

We asked ten Claude Opus 5.5 agents to devise a faster shortest-path algorithm and prove it in Lean. Within 15 hours, they produced C-HD: a formally verified improvement over the published bounds. https://t.co/rxLKycsPoD

This is the first release in our open-source week: runtime dynamic compression and the private beta for bitsandbytes2. bnb2 offers a new level of compression that brings large models to smaller devices. The bnb2 private beta has started. We offer a limited number of people access…

Today, we’re announcing Ternary Bonsai 2 27B. Based on Qwen3.8 27B, Bonsai 2 27B is 9x smaller than its full-precision counterpart while retaining 98.2% of its aggregate benchmark performance. Two months after the first Bonsai 27B release, the biggest change is quality. The footp…

Full walkthrough — the whole flow, what runs where, why the shape has to be what it is, honest about what's built vs. rough: https://t.co/UxAo5OGSLP Fourth post in a four-post arc: vision → framework → principle → implementation. If you're building in this space, curious what you…

𝗔𝗱𝗮𝗺𝗪 𝘃𝘀. 𝗠𝘂𝗼𝗻: α, 𝗢𝘃𝗲𝗿𝗳𝗶𝘁𝘁𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗠𝗲𝗺𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻. A quick update on our WeightWatcher experiments comparing AdamW with the newer Muon optimizer. We trained a single-head nanoGPT model with both optimizers across five seeds. We then used WeightWatcher to study the weight-matrix sp…

New from @Meta Engineering: FlashAttention-4 extended with MXFP8 support for @nvidia Blackwell—from forward and backward kernels to fused quantization and jagged cross-attention. The team developed an end-to-end jagged module with fused quantization, FP8 activation and compute wh…

On building an agent harness from scratch. Got so many questions about where to get started. My short guide (feed it to your agent): If you really want to learn harnesses well, it's worth building one from scratch using a programming language (TypeScript or Python) of your choice…

For those already asking, the custom harness is basically a meta harness I build that connects to these different harnesses. It handles routing, model switching, orchestration, context handoffs, memory, compaction, and all that. As I share more about harnesses, I will spend some …

A new open release from @NVIDIAAI has achieved an IMO gold-medal score. Its Nemotron 3 Ultra system scored 30/42 at the 2026 International Mathematical Olympiad using natural-language proofs only. No formal theorem prover, external tools, or internet access. Three checkpoints gen…

Fugu Ultra v2 by the numbers. Full Results: https://t.co/vjyNB5lWGp • #1 on 5 out of 8 hard benchmarks (including DeepSWE, Chartography, Toolathon) • DeepSWE: 74.3 • Chartography: 48.3 vs Opus 5 at 27.3 Achieved without Fable 5, Fable 5.1, or GPT-6-Astra in the agent pool.

we RL trained a 4B VLM to play GeoGuesser > environment, dataset, training recipe, evals, and code are all open-source. > runs on a single A100 > beat gpt 5.4 mini, haiku, qwen 3.5 122B and came close to sonnet on evals https://t.co/QqGyompcW1

I put together a mega write-up on GPT-6 Astra & looped transformers. How looped transformers / recurrent depth works, cost-tradeoffs, whether it hides reasoning traces, with lots of figures and a tour of recent looped transformer research. https://t.co/VR6HaPFVXH

Reasoning from scratch round 3: This time, I cover generating a verifier for... a) ...evaluation (base model versus any future model improvement) b) ...the reinforcement learning with verifiable rewards (RLVR) training later on 00:00 Introduction 01:21 Four approaches to LLM eval…

Introducing FlyOCR 🪰 - I trained a fly brain to read a PDF It uses the full MaleCNS v1.0 fruit fly connectcome. The architecture is inspired by doomfly by @nftechie_ The fly splits a pdf image into individual glyphs, maps pixels into receptor activations, runs simplified current-…

Reasoning from scratch round 2: In this video, I cover the text generation process in LLMs and KV caching (to prepare the base model before adding reasoning techniques in the upcoming ones). 00:00 Introduction and reasoning model demo 01:55 How to work through the book 05:00 Chap…

Astra is shockingly good in reasoning! I benchmarked it on induction, and it almost saturated it with 88%. Fable 5.1, by comparison, is at 33%. The final numbers will actually go up: I am running a residual batch run on the non-evaluable; the results here reflect one batch run at…

We benchmarked GPT-6 Astra on hard document parsing and extraction tasks. It is the best frontier model for one-shotting document extraction over short and medium-sized documents: 📊 97.2% over short documents, achieving a new SOTA on our benchmark 📊 90.6% over medium documents Bu…

Banger paper from Google DeepMind. Every AI weather model so far has been trained and initialized on analysis data, which is itself the output of another model. That means the forecast inherits whatever biases the analysis carries, and it cannot use a new satellite observation di…

@SamuelMinouri Nice, glad you found it helpful. In addition, I also have a from-scratch KV Cache article here if useful: https://t.co/XHADskY0GI

@paulgamble100 Indeed, the model can now perform test-time search in latent space instead of token space. It can trade off reasoning from one mode to the other.