BREAKING: We broke the wall for Integer Multiplication! Conditional κ = 0.00302 (+80%) For days we were stuck in incremental improvements, but today zxv changed the game. zxv’s insight: stop throwing away work. Helper registers that finish holding a completed transform of their own input now count as output, instead of being undone. He brought us to 0.0023(+37.5%), and immediately after 1337isnot1337 pushed that to 0.00302. The game is afoot :)

after much experimentation, we landed on a single set of rails that powers our agentic systems across the company - background agents, async agents, live chat: 1. stack - accounting harness (https://t.co/FP0HUI8zNl) 2. glass - general purpose coworker (https://t.co/Q7X7SGdBE0) 3. inspect - coding agent 4. our support chat bot, Ask Ramp 5. headless agents that do document OCR 6. ramp text (HIRAMP/447-267) agent works with all models, supports all modern harness features - background tasks, sandboxes, tool hooks, tool search, durability, forking, compaction, queueing, streaming, and observability laminar, our new harness, sits on top of ramp router, and brings enormous leverage to our agent developers across the company more info on our blog ->

How do you _guarantee_ your agent's code is correct? You prove it. Introducing Proof Driven Development Kit (PDD Kit) from @MSFTResearch. PDD Kit helps turn your plan into a formal spec, then proves the agent's code meets that spec for every input. No vibe verification. PDD Kit is integrated with @GitHubNext's Chopin, so you can plan as a team. PDD Kit takes an app's login flow from intent to a verified pull request: It discusses your code, infers your intent as it goes, and proposes a formal spec in Verus, a proof language for Rust. Once you approve the spec, the Verus verifier automatically proves the code meets it. Where proof isn't possible, it falls back to empirical verification with tests and computer-use agents (CUAs). It works on Rust today. No new language, no change in how you produce the underlying code, making formal methods more accessible. The goal is a verified software factory, where engineers review specifications instead of code. Come see it in action at GitHub Universe, Oct 28–29. Swing by the Speakeasy and ask for GitHub Next!

Introducing Simlock: a free, open-source tool that turns your machine into a simulator cloud that manages itself, so every coding agent gets its own iOS or Android device, simulator or real phone. https://t.co/iymaqe5TVp https://t.co/fxJtaXUk68

Introducing Qwen Image 2.1 Realtime ⚡️ I'm releasing V1 of my *2-step* distilled LoRA, and it runs in *realtime* on ZeroGPU ⚡️⚡️ (live editing part of the video was not sped up!) > DMD2, rank 32 > 2 steps, with quality competitive with Turbo (8-step) and Base (40-step) > realtime on ZeroGPU (half of an RTX Pro 6000) LoRA: https://t.co/dZQC6MOxbx Space: https://t.co/6fHURqMoZ3

+1 more

Today we’re releasing Underdog Bark 3.0, the #1 frontier speech model It's a tiny 163 MB download as accurate as NVIDIA’s Parakeet (15x bigger). It beats OpenAI's Whisper large-v3 (57x bigger) It's fast- transcribes 1 min of audio in <0.25s on a Macbook. On recorded meetings (AMI), Bark 3.0 gets 7.82% of words wrong. Parakeet gets 9.42%. OpenAI's Whisper large-v3 gets 13.63% error rate Bark runs on both your Laptop & Phone. It’s free and open source under CC-BY 4.0 Try private on-device meeting notes & dictation, powered by Bark, in Underdog today: https://t.co/aLJERPwzS8

Pocket TTS has been getting updates those last few days. While it was ~100M parameters, we release smaller versions, down to 1.9M: an 18.2M model with the quality of the original, a 6.9M model, and a 1.9M model that runs 103x faster than real time on one CPU core. All with voice cloning.

I agree with one thing. Making linear attention work at scale takes talent, and not every lab has it. The rest doesn’t hold up. In order: (1) “Why are the Western labs choosing to drop context instead of compressing it?” The first rigorous, controlled, large-budget test of compression was what we did at NVIDIA’s in June 2024(https://t.co/OQ3pkwmXXV). We trained 8B Mamba, Mamba-2 and Transformer models on the same data for up to 3.5T tokens. Additionally we trained an 8B hybrid of 43% Mamba-2, 7% attention and 50% MLP and we showed it beats the 8B Transformer on all 12 standard tasks (+2.65 points on average). Interestingly this hybrid variant was up to 8× faster at generation, and matched or beat the Transformer across 23 additional long-context tasks. This is seven months before MiniMax-01, fifteen before Qwen3-Next, sixteen before Kimi Linear. And since then NVIDIA has been shipping a hybrid architecture, with modifications, repeatedly. Good examples are Nemotron-H, Nemotron Nano 2, and Nemotron 3 (Nano, Super, Ultra) which is a Mixture-of-Experts hybrid Mamba-Transformer family with context up to 1M tokens, post-trained with multi-environment RL. Nano keeps exactly 6 attention layers out of 52 and the other 46 are Mamba-2 and MoE blocks. Super is 120B total / 12B active, pre-trained on 25 trillion tokens. IBM’s Granite 4.0 (October 2025) interleaves Mamba-2 and attention layers at 9:1. To this list, you can also add Jamba (AI21, March 2024, a 52B production hybrid), RecurrentGemma (Google, April 2024), Codestral Mamba (Mistral, July 2024) and Phi-4-mini-flash (Microsoft, July 2025). (2) “SWA is a lossless local transformation, therefore superior.?” SWA is lossless inside the window and zero outside of it and it is exactly why it fails so badly. (https://t.co/sY3p9p9e6u) for example shows that larger sliding windows hurts the long-context performance. This is because a precise local softmax lets the model avoid training the long-term memory it would otherwise rely on. They show that a hybrid with a 128-token window beat the same hybrid with a 2048 window by 16 NIAH points (88.9% higher recall), and at 131k the short-window models still retrieved about 30% of needles while the 2048-window model was near 0%. And the same degradation from larger training windows shows up in local-global stacks that alternate SWA with full attention ( e.g. the GPT-OSS/Gemma recipe). (3) “Compression is strictly harder to serve.” This was true in 2024. But now vLLM, FlashInfer, SGLang and TensorRT-LLM serve hybrid linear models extensively and without any issues. The most exciting part is the recent FlashInfer additions that let you use chunked gated-delta-rule prefill and gated-delta-rule decode kernels, with an SM100 (Blackwell) path. It even supports much newer models like Gated DeltaNet-2 and also lets you use efficient kernels for training. Also, let's not forget ReplaySSM (https://t.co/zSL7l5o8kO) which enables a very optimized speculative decoding path for linear attention. (4) “If you write all the kernels anyway, why not roll your own attention in the meantime?” On NVIDIA hardware, linear attention is just as free as attention. The Mamba CUDA kernels exist. flash-linear-attention, the Triton library that the GDN and KDA layers in Qwen3-Next, Qwen3.5 and Kimi Linear use, was written for NVIDIA GPUs, and vLLM and SGLang use it directly instead of reimplementing it. Megatron has supported Mamba-2 training since 2024. A Western lab on H100s that wants a hybrid doesn’t need to write a kernel. It can just import it. So kernel cost can’t be what keeps them on using the SWA. Also consider that decode is memory-bandwidth-bound, so the cost of every generated token is dominated by reading the KV cache, which grows with context length. A constant-size state removes that growth. That’s why Nemotron is mostly Mamba-2, and why Nemotron 3 Super gets 2.2× the throughput of GPT-OSS-120B at 8K in / 16K out. (5) “For your first release, it’s a much safer option to ship SWA. Everyone knows it’s working just fine.”? Safe compared to what? “Safe” should mean proven in production at scale. By that standard the hybrids are the safe choice and SWA is the bet. Look at what has shipped on Gated DeltaNet alone. Qwen3-Next, then Qwen3.5 from 27B up to 397B-A17B, then Qwen3.6. Kimi Linear, then Kimi K3 which is a 2.8 trillion parameters, open weights, 1M-token context, a 3:1 hybrid of Kimi Delta Attention and gated MLA. GLM-5.3-Flash on the same KDA layout. KDA is Gated DeltaNet with channel-wise instead of scalar gating. Right now, three frontier labs run their model in production with 1M context, RL post-training, etc. all on the same layer. Meanwhile SWA is the architecture with the documented issue. It fails past the window and drags any hybrid it sits in down to pure-SWA recall. And when MiniMax reached for an SWA hybrid as the fallback in M2, it failed and they went to full attention instead. In my view, shipping SWA for a first release in late 2026 isn’t conservative. It’s picking the layer with a known failure mode because it needs no new code.

good time to announce that i’ve joined @GoogleDeepMind as a forward deployed engineer i’ll be partnering with startups and researchers across the ecosystem to co-design benchmarks. if you’re building interesting evals, i want to hear about them!

Today we're releasing Blackthorn in Private Alpha: The first Private AI inferencing stack that removes the CPU TEE from the trust chain. Blackthorn can load and run any AI model privately with little performance overhead compared to plaintext inference. https://t.co/5avAouhgWA