@BowsersaurusRex @peterwildeford I think we're maybe talking past each other. Let me make a specific mechanistic claim: I claim that when the model writes a poem, there are activations on the tokens at the end of the line which represent candidate targets for the end of the next line. https://t.co/fyxkKMJIEh https://t.co/RmFIf1jcFA

@BowsersaurusRex @peterwildeford We know this is true, because we can modify those activations and change the next line correspondingly. https://t.co/sDoFDk9CA5

We did a very careful study of 10 optimizers with no horse in the race. Despite all the excitement about Muon, Mars, Kron, Soap, etc., at the end of the day, if you tune the hyperparameters rigorously and scale up, the speedup over AdamW diminishes to only 10% :-( Experiments are made possible by Marin (https://t.co/UgEjGM0HPY); anyone developing new optimizers: please come try your method on this benchmark!

Excited to share what friends and I have been working on at @Standard_Kernel We've raised from General Catalyst (@generalcatalyst), Felicis (@felicis), and a group of exceptional angels. We have some great H100 BF16 kernels in pure CUDA+PTX, featuring: - Matmul 102%-105% perf of cuBLAs in 100 lines of code - Attention 104% perf of FlashAttention3 in 500 lines - Fused Llama3 FFN 120% perf of PyTorch (gpt-fast) Reach out if you want to work on AI kernel gen with us!

How does ChatGPT not know that Donald Trump is the current US President? I asked ChatGPT: "Can you give me the top news stories of the day?" The exchange below is pretty bonkers -- even when I pushed back, it still did not acknowledge he is the current US President https://t.co/OkfKKG1uGP

A must read indictment of generative AI. Like seriously, stop what you’re doing and read this essay. It doesn’t nibble around the edges. https://t.co/75R9eZG31u

Taking 𝘐𝘧 𝘈𝘯𝘺𝘰𝘯𝘦 𝘉𝘶𝘪𝘭𝘥𝘴 𝘐𝘵, 𝘌𝘷𝘦𝘳𝘺𝘰𝘯𝘦 𝘋𝘪𝘦𝘴 to the streets… https://t.co/EVsNnzJyHC

Is If Anyone Builds It, Everybody Dies by @ESYudkowsky and @So8res “the most important book of the decade” or just “Scientology”? Don’t just blow the book off; take these potentially catastrophic issues seriously. But don’t believe every word, either. @TheTLS I dissect what they got right, and what they got wrong. Link below.

Dissecting rather than dismissing If Anyone Builds It, Everybody Dies @TheTLS : https://t.co/oKZF5Gvl8Q

This is the clip so the audience can see it in full. https://t.co/jzSvBJg0PG

Y'all fuck with ilya merch? https://t.co/rISV4ABGit

Brand exploration for OpenAI with Sam Altman, February 2023 Two logo concepts – Circle and Monogram – alongside broader exploration for ChatGPT across brand and product. More details and link below. https://t.co/y0gWzAqlPV

+2 more

AI challenges t he dominance of Google search https://t.co/Y7jJ7YY58c @sbearne @bbcnews

AI challenges t he dominance of Google search https://t.co/Y7jJ7YY58c @sbearne @bbcnews

‘I have to do it’: Why one of the world’s most brilliant AI scientists left the US for China https://t.co/qcVOQH4fQJ

‘I have to do it’: Why one of the world’s most brilliant AI scientists left the US for China https://t.co/qcVOQH4fQJ

Meta struggles to decouple from Chinese supplier of AI smart glasses https://t.co/aZFcI2JjXv @MsHannahMurphy @EleanorOlcott @ft

Meta struggles to decouple from Chinese supplier of AI smart glasses https://t.co/aZFcI2JjXv @MsHannahMurphy @EleanorOlcott @ft

AI companies want copyright exemptions for NZ creatives, the market is their best protection https://t.co/G4ZZPYTWQH @ConversationEDU

Google owner reveals £5bn AI investment in UK ahead of Trump visit https://t.co/pBbUnenDOY @google @faisalislam @bbcnews

Google owner reveals £5bn AI investment in UK ahead of Trump visit https://t.co/pBbUnenDOY @google @faisalislam @bbcnews

Vibe coding has turned senior devs into ‘AI babysitters,’ but they say it’s worth it https://t.co/HrFWgM9wmn @DominicMadori @techcrunch

Vibe coding has turned senior devs into ‘AI babysitters,’ but they say it’s worth it https://t.co/HrFWgM9wmn @DominicMadori @techcrunch

‘I love you too!’ My family’s creepy, unsettling week with an AI toy https://t.co/FCT72XebCL

‘I love you too!’ My family’s creepy, unsettling week with an AI toy https://t.co/FCT72XebCL

Facebook owner Meta unveils new AI-powered smart glasses https://t.co/N9uROyQ1dd @lilyjamali @bbcnews

Jensen Huang 'disappointed' by reported China Nvidia chip ban https://t.co/ymEJunHXDj @faisalislam @bbcnews

AI CEO says technology ‘moving very quickly,’ could soon replace more jobs https://t.co/2BRDBik9OL

US tech groups answer Starmer’s call for AI infrastructure spending https://t.co/4NRwwSOjkg @tim @RachelMillardAS @AnnaSophieGross @ft

AI Chatbots Are Not Therapists: Reducing Harm Requires Regulation https://t.co/zKk76Z8KAA @techpolicypress

AI can forecast your future health just like the weather https://t.co/XTm41WyFCl @JamesTGallagher @bbcnews

What is Microsoft's new AI supercomputer in Loughton, Essex? https://t.co/UePEDMFdCu @bbcnews