We gave GPT-6 Astra and Claude Fable 5.1 open-ended AI research questions from unpublished papers, with a $10,000 budget each, and had the original authors grade the papers they produced. Compared to our first tests with Claude Opus 4.8 in July, we saw meaningful improvements in experimental design and resource management, but persistent limitations in creative problem solving, judgment, calibration, and communication; both papers were rejected by our experts. https://t.co/n1OMKf4M7T

One thing I like about @paperswithcodee is how it automatically surfaces trending benchmarks for me e.g. right now the top benchmark for image understanding is Chartography, which includes 100 professional chart-understanding tasks from @HelloSurgeAI I had no idea this benchmark existed, but both Anthropic and DeepSeek are reporting evals on it for their new models Find it here: https://t.co/pxwrAzgIj0

This is the topology of the tinybox red kimi, frontier models at home at usable speeds for a good price. tinygrad has a driver to directly control the NVMe drives from the AMD GPUs. https://t.co/YlHhaD7Kpl

Before he logs off, @mitchellh sends Codex agents to compare tools and prototype features he’s still figuring out. He picks up from there the next morning. https://t.co/AXVctw8Es5

I trained a specialist decision model for curating web-scale data. It's small (380M, fine-tuned from UltraBERT-380M), so it runs relatively cheaply at scale: 1M FineWeb pages took 19 minutes and ~$5 on @huggingface Jobs. It thinks a quarter of those pages are product pages. 6% passed a simple "educational, not spam" filter, and 1 page in 1,020,000 got "outstanding". I've shared the model, the training data (with the source, labeller, and licence for every row), and the full recipe. I hope more people share how they prepare data for training open decision models! We've had some cool open model releases. Time for some more data releases! https://t.co/IdHCqnmnP1

too many benchmarks are too sanitized. they test if models can solve well-defined problems when you tell them exactly what problem is. but figuring out the problem is half the job and all the fun infusing the chaos of reality into our benchmarks (and figuring out how to grade them nonetheless) is what our Evals team does every day. ~~introducing DAYJOB: TAX, the next in our economically valuable 9 to 5 series~~ one messy example: Anna asks her tax advisor whether to sell her house in Oct or Jan, to optimize her taxes. a good advisor tells her that both dates qualify for a $250K exclusion in capital gains... ...but an even better advisor proactively warns her that she could lose the exclusion if the sale gets inevitably delayed past next August. (Opus mentioned the August deadline 5 out of 5 times. Astra never did.) (maybe your Opus bill is worth it after all!) knowing tax rules isn't enough. great tax agents should give you *advice* too, and have the professional judgment and experience to handle everything you didn't know to ask. we built DAYJOB: TAX (and our entire DAYJOB Knowledge Work series) full of the messy holes of reality that even the smartest agents still don't know how to fill in.

Interesting finding about reasoning models and CoT. They find that reasoning models can control their chain of thought much better than earlier tests suggested. Earlier evaluations put the constraint in the user prompt and found very low controllability. Researchers put the same constraint inside the chain of thought as a self-instruction, and models followed it far more often. A generic reminder to follow the user's constraint also helped a lot. The self-instruction can be triggered from the user prompt alone, so no access to the reasoning channel is needed. Paper: https://t.co/fyagSbVT8M Chat with Paper: https://t.co/90LdW0qwdW

How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference New paper from Meta. "Our experiments span models with up to 1B parameters, trained on up to 155B tokens using up to 1.07 × 10^21 training FLOPs, with up to 32 loops. We find that looping the full transformer generally gives the strongest quality at a fixed parameter budget, whereas looping a middle block of layers gives a better return when training FLOPs are constrained." "We compare exact weight sharing across 2 loops with small loop-specific low-rank adapters that allow each loop to learn a slightly different transformation. These adapters provide only modest gains when few loops are used, but become consistently more useful as the same weights are reused across more loops" "Naively applying Parallel Looped Transformers (PLT) in our setting does not preserve the gains from looping: quality can degrade as more loops are added. We find that preserving the current-token representation, separately normalizing it and the recurrent state, and controlling how they are combined substantially improves PLT."

Today we're launching Mouse-1. A fast decision model built specifically for agent harnesses. https://t.co/zbIMGwkztp

Perplexity Flash takes the crown! ⚡ In our tests, it matched Wald's accuracy at nearly twice the speed on a 16 GB Mac mini. Synthetic agent decisions, real email threads, prompt injections: see why Flash is our new pick for email-completion checks. 👇 https://t.co/w7qK7mrGQN