Really enjoyed reading the Microsoft MAI-Thinking-1 "Building a Hill Climbing Machine" paper. Amazing they publicly released all the info needed to train a frontier model, down to hparams. I also thought this was pretty telling: - pre-training: 30 trillion tokens - mid-training (SFT on STEM/math/code data): 3.55 trillion tokens - RL post-training: 150 billion tokens. Looks like @ylecun was right all along with the cake analogy. Obviously I still think something like RL (optimizing for long term goals) is fundamental to what we think of as intelligence. But it's not the volume of learning signal, it's the optimization on top of an already reasonable predictive model.

@ClementDelangue @Dan_Jeffries1 Everyone, please join Project Tapestry https://t.co/5MOgouVplV

@ClementDelangue @Dan_Jeffries1 Everyone, please join Project Tapestry https://t.co/5MOgouVplV

The videos from the “Frontiers of Embodied AI” meetup at ETHZ from a few weeks back are now available. Speakers: Jitendra Malik, Vladlen Koltun, Yann LeCun, and Shuran Song Hosted by Marc Pollefeys YouTube playlist: https://t.co/IfU9owsa1o https://t.co/dNiH3OfBYm

+2 more

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development "Any topic related to building pretraining pipelines, distributed training infrastructure, or ML accelerator design... may have limited effectiveness through Claude via methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning." Not only do they get to decide what you use LLMs for in research, but this also enables them to silently intervene in your research without you knowing. This sets a dangerous precedent. If a model refuses openly, users can understand the boundary. If a model falls back to another model, users can still evaluate the difference. But if a model silently modifies or weakens its own answers while still pretending to help, researchers lose the ability to know whether a failed result came from their own idea, their implementation, or an invisible intervention by the model provider. That is not safety. Safety policies should be transparent, auditable, and user-visible. On top of that, the people most harmed by this are not the largest labs with massive teams and proprietary infrastructure. It is the independent researchers, academic groups, startups, and open-source builders who rely on public tools to compete, innovate, and pioneer AI for everyone else.

Yann Lecun published the most heretical AI paper of the year. He opens by arguing Magnus Carlsen isn't good at chess and only gets more unhinged from there. The Turing Award winner and his co-authors dropped a paper demanding the AI industry abandon its biggest obsession, AGI. Right now, everyone from Silicon Valley CEOs to politicians assumes AGI is the ultimate goal. A machine that can do everything a human can do. LeCun argues that this entire concept is a biological illusion. Humans do not possess "general" intelligence. We are highly specialized biological machines, tuned by evolution simply to survive in the physical world. We only think our intelligence is "general" because we are completely blind to the millions of cognitive tasks we are incapable of comprehending. Which brings us to the chess argument. Magnus Carlsen is the greatest human chess player in history. But compared to a modern computer? He is fundamentally terrible. Our belief that Carlsen is "good" at chess is pure human-centric bias. He isn't objectively good. He's just better than the rest of us, who are biologically awful at it. LeCun says we need to stop building AI to mimic human generality. Instead, he proposes a new North Star: SAI. Superhuman Adaptable Intelligence. Instead of trying to build a machine that mimics our flawed, biologically-limited brains, we need to embrace extreme specialization. SAI is about the speed of adaptation. It is an intelligence that can learn to exceed humans at any specific, economically important task. More importantly, it is designed to fill the vast skill gaps where humans are fundamentally incapable. Things like managing global energy grids in real-time. Or predicting complex molecular structures. The entire AI industry is obsessed with building a digital reflection in our own image. LeCun's paper is a brutal wake-up call.

LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). https://t.co/cOTeMUr4py

New paper 🧵 We show that dynamic short convolutions consistently improve Transformers across scales. We make these gains practical with an efficient parameterization and custom Triton GPU kernels. The improvements carry over to MoEs and linear attention variants (Mamba-2/GDN). https://t.co/Py6isYX0LK

In film, "we'll fix it in post" is what you say when something went wrong on set and you don't want to redo it. AI research has made it our entire methodology: train the model, then patch whatever comes out. Our new ICML oral argues this can't be the basis of a science of AI. 🧵 https://t.co/ok11oGRhUQ

We ground discussion in the history and philosophy of science. What did it take for other fields to move from cataloging phenomena to predicting and controlling them? AI can learn from that playbook. https://t.co/rWWMVVFlgn

A common issue with position papers is that they leave the reader wondering “okay, but what should I actually do”? To address this we provide open problems on a wide variety of topics throughout to illustrate our perspectives and guide future research https://t.co/2FSgK95W4K

+1 more

@guilhermeotina Yes, if we said that we would be very silly. But that's not what we're talking about. Scaling laws, grokking, and induction heads are some of the best examples of the kind of work we are advocating for. https://t.co/dUfGEGXgp4

@typewriters Literally put it on my calendar for next year https://t.co/tSz9DGVr03

Highlighting the new WebGPU backend in llama.cpp/ggml The work to bring full-fledged WebGPU support in llama.cpp started about an year and a half ago. It has been lead by @reeselevine and team at USCS. For more information, checkout the interactive blog and paper in the quoted post. Here are 2 excerpts from the paper, summarizing the implemented software architecture.

More info and discussion: https://t.co/w9rujz5SOY

These are some of my LLM assisted contributions from the past month. Nothing amazing, but I'm slowly getting better at it. Atm, using Qwen3.6 27B exclusively. For hardware - switching between M2 Ultra and RTX 5090. Both are good options, though after using the RTX and going back to the Mac, it always feels like a snail. Yet for most tasks, I feel like both hardware can do the job comfortably.

Build on-device personal AI agents on Windows PCs with new tools from NVIDIA and Microsoft, including secure sandboxing, faster local inference, multi-GPU support, and RTX acceleration for Windows AI APIs. Read the technical blog: https://t.co/vNIsEded46 https://t.co/4bPWPDERJO

Highlighting recent advances in multi-GPU and tensor parallel support in llama.cpp Over the last few months llama.cpp maintainers and engineers from NVIDIA collaborated to improve the multi-GPU performance in ggml. This resulted in significant performance gains on RTX systems and laid the groundwork for hardware-agnostic tensor parallelism in ggml. For more information on this and other advancements in the low-level inference engine of llama.cpp, check the technical blog by @NVIDIARTXSpark below

Building super fast experiences with Gemma just got easier. Gemma 4 MTP is now officially merged into llama.cpp. Developers can now pair MTP with Gemma 4 QAT for a fast, lightweight setup. https://t.co/CIynVMYuZm

i just beat @GoogleDeepMind's turboquant introducing Shard. 10x KV cache compression on Llama-3.1-8B. zero quality loss - 10x @ 8K context, 11.2x @ 32K - NIAH recall 1.000 across 4K-32K - LongBench Δ ≈ 0 vs FP16 turboquant tops out at 4-6x at the same quality. we doubled it. read more: https://t.co/PAV5WdAzN6 @kirrithan

AI now helps doctors read X-rays, CTs, and MRIs. But once it's deployed in a hospital, almost no one can tell if it's still accurate. Lattice Health watches deployed imaging AI and flags it the moment it starts to slip. Congrats on the launch, @sparkcpark! https://t.co/VWndL8ja8E

@_DavidLKing_ @VirginiaLConn -57 https://t.co/lgmsXLWYkG

Submissive and Breadable https://t.co/l7gZZQh4f5

@16kbps @atomicthumbs There's also this EXTREMELY cursed "standard" https://t.co/dwuk6WFlxQ https://t.co/RGRAc9XaHR

@retronouns https://t.co/Ew7EqgSse0

Jay Mohr's anecdote about meeting Christopher Walken is the kind of anecdote you could only wish for. @FighterNtheKid https://t.co/OoykZtfd52

And the winner is Ali Smith’s Gliff. A wonderful novel. @DublinLitAward https://t.co/lsRA3JNfJf

Spotify Introduces AI-Generated Personal Podcasts https://t.co/NSHw1C6OWW

Playgrounds are much cooler these days... This is a space themed park in Erie CO. In the background you'll see a path with solar system facts on engraved stones. Behind me is a fake half Moon you can climb. https://t.co/ohlfXxkSBr

Put down your phone and be present with those around you or go for a walk... > JOMO - digital wellbeing @google https://t.co/XKuxWkma8M

A recent post from @joeerl (author of erlang) has me thinking about web-futures. I reflect on how far the web has come, and how much utility there is now - but he is right, it should go *much* further, and decentralization & persistence is key. https://t.co/DqvYo2AQVb

When you have plumbing... You must knit it. https://t.co/TPRBhkF6kG