🐦 Twitter Post Details

Viewing enriched Twitter post

@SwayStar123

I'm implementing a tiny transformer on tinyshakespear. The 2d matrices went to muon and 1d to adam. The model was still learning and generating some real words. But turns out my muon implementation was bugged and was no-oping. So 99.2% of my model was frozen at init. Adam still managed to tweak those biases into having the model still output some real words. Oh and I forgot the positional embeddings too. It's kinda crazy how you can have the shittiest implementation and a neural network still manages to learn

📊 Media Metadata

{
  "score": 0.42,
  "score_components": {
    "author": 0.09,
    "engagement": 0.0,
    "quality": 0.12,
    "source": 0.135,
    "nlp": 0.05,
    "recency": 0.025
  },
  "scored_at": "2026-08-31T19:03:08.516344",
  "import_source": "api_import",
  "source_tagged_at": "2026-08-31T19:03:08.516356",
  "enriched": true,
  "enriched_at": "2026-08-31T19:03:08.516357"
}

🔧 Raw API Response

{
  "type": "tweet",
  "id": "2094410661661220983",
  "url": "https://x.com/SwayStar123/status/2094410661661220983",
  "twitterUrl": "https://twitter.com/SwayStar123/status/2094410661661220983",
  "text": "I'm implementing a tiny transformer on tinyshakespear. The 2d matrices went to muon and 1d to adam.\n\nThe model was still learning and generating some real words.\n\nBut turns out my muon implementation was bugged and was no-oping. So 99.2% of my model was frozen at init.\nAdam still managed to tweak those biases into having the model still output some real words.\n\nOh and I forgot the positional embeddings too.\n\nIt's kinda crazy how you can have the shittiest implementation and a neural network still manages to learn",
  "source": "Twitter for iPhone",
  "retweetCount": 4,
  "replyCount": 1,
  "likeCount": 63,
  "quoteCount": 0,
  "viewCount": 4428,
  "createdAt": "Mon Aug 31 13:03:08 +0000 2026",
  "lang": "en",
  "bookmarkCount": 26,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "2094410661661220983",
  "displayTextRange": [
    0,
    280
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "SwayStar123",
    "url": "https://x.com/SwayStar123",
    "twitterUrl": "https://twitter.com/SwayStar123",
    "id": "3265925444",
    "name": "sway",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1816746261401260032/q97HRswg_normal.png",
    "coverPicture": "https://pbs.twimg.com/profile_banners/3265925444/1721981334",
    "description": "",
    "location": "",
    "followers": 1874,
    "following": 662,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Thu Jul 02 09:55:12 +0000 2015",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 10712,
    "hasCustomTimelines": true,
    "isTranslator": false,
    "mediaCount": 389,
    "statusesCount": 1750,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [
      "2000826634409320918"
    ],
    "profile_bio": {
      "description": "Vegan btw, working on diffusion models 日本語できる, 中文学习中\n\nBuilding https://t.co/JKM8o4UAt0, working at @LumaLabsAI",
      "entities": {
        "description": {
          "urls": [
            {
              "display_url": "languagespeedrun.com",
              "expanded_url": "http://languagespeedrun.com",
              "indices": [
                63,
                86
              ],
              "url": "https://t.co/JKM8o4UAt0"
            }
          ],
          "user_mentions": [
            {
              "id_str": "",
              "indices": [
                99,
                110
              ],
              "name": "",
              "screen_name": "LumaLabsAI"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "swaystar123.github.io",
              "expanded_url": "https://swaystar123.github.io",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/3kDtmNbimY"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {},
  "card": null,
  "place": {},
  "entities": {
    "hashtags": [],
    "symbols": [],
    "urls": [],
    "user_mentions": []
  },
  "quoted_tweet": {
    "type": "tweet",
    "id": "2093623060016275908",
    "url": "https://x.com/iScienceLuvr/status/2093623060016275908",
    "twitterUrl": "https://twitter.com/iScienceLuvr/status/2093623060016275908",
    "text": "Karpathy's recipe for training neural networks is still relevant today.\n\nthis lesson in particular is one we've been feeling very viscerally recently...\n\nneural network training can sometimes be very resilient and you may not realize there's an error for a very long time... https://t.co/51qIY0oKZd",
    "source": "Twitter for iPhone",
    "retweetCount": 10,
    "replyCount": 3,
    "likeCount": 172,
    "quoteCount": 1,
    "viewCount": 12924,
    "createdAt": "Sat Aug 29 08:53:29 +0000 2026",
    "lang": "en",
    "bookmarkCount": 120,
    "isReply": false,
    "inReplyToId": null,
    "conversationId": "2093623060016275908",
    "displayTextRange": [
      0,
      274
    ],
    "inReplyToUserId": null,
    "inReplyToUsername": null,
    "author": {
      "type": "user",
      "userName": "iScienceLuvr",
      "url": "https://x.com/iScienceLuvr",
      "twitterUrl": "https://twitter.com/iScienceLuvr",
      "id": "441465751",
      "name": "Tanishq Mathew Abraham, Ph.D.",
      "isVerified": false,
      "isBlueVerified": true,
      "verifiedType": null,
      "profilePicture": "https://pbs.twimg.com/profile_images/2058414948255211520/8jVMerrR_normal.jpg",
      "coverPicture": "https://pbs.twimg.com/profile_banners/441465751/1779599917",
      "description": "",
      "location": "",
      "followers": 90360,
      "following": 1525,
      "status": "",
      "canDm": true,
      "canMediaTag": true,
      "createdAt": "Tue Dec 20 03:45:50 +0000 2011",
      "entities": {
        "description": {
          "urls": []
        },
        "url": {}
      },
      "fastFollowersCount": 0,
      "favouritesCount": 131724,
      "hasCustomTimelines": true,
      "isTranslator": false,
      "mediaCount": 2991,
      "statusesCount": 21402,
      "withheldInCountries": [],
      "affiliatesHighlightedLabel": {},
      "possiblySensitive": false,
      "pinnedTweetIds": [
        "2088229088322560106"
      ],
      "profile_bio": {
        "description": "CEO @SophontAI |\nFounder @MedARC_AI |\nPhD at 19 (2023) |\nex Research Director Stability AI | \nBiomed. engineer @ 14 |\nTEDx talk➡https://t.co/xPxwKTq6Qb",
        "entities": {
          "description": {
            "urls": [
              {
                "display_url": "bit.ly/3tpAuan",
                "expanded_url": "https://bit.ly/3tpAuan",
                "indices": [
                  128,
                  151
                ],
                "url": "https://t.co/xPxwKTq6Qb"
              }
            ],
            "user_mentions": [
              {
                "id_str": "",
                "indices": [
                  4,
                  14
                ],
                "name": "",
                "screen_name": "SophontAI"
              },
              {
                "id_str": "",
                "indices": [
                  25,
                  35
                ],
                "name": "",
                "screen_name": "MedARC_AI"
              }
            ]
          },
          "url": {
            "urls": [
              {
                "display_url": "sophont.med",
                "expanded_url": "https://sophont.med",
                "indices": [
                  0,
                  23
                ],
                "url": "https://t.co/MvROZZW1Zg"
              }
            ]
          }
        }
      },
      "isAutomated": false,
      "automatedBy": null
    },
    "extendedEntities": {
      "media": [
        {
          "allow_download_status": {
            "allow_download": true
          },
          "display_url": "pic.x.com/51qIY0oKZd",
          "expanded_url": "https://x.com/iScienceLuvr/status/2093623060016275908/photo/1",
          "ext_master_playlist_only": [],
          "ext_media_availability": {
            "status": "Available"
          },
          "ext_playlists": [],
          "features": {
            "large": {
              "faces": [
                {
                  "h": 161,
                  "w": 161,
                  "x": 935,
                  "y": 585
                }
              ]
            },
            "orig": {
              "faces": [
                {
                  "h": 167,
                  "w": 167,
                  "x": 966,
                  "y": 605
                }
              ]
            }
          },
          "id_str": "2093622310154518528",
          "indices": [
            275,
            298
          ],
          "media_key": "3_2093622310154518528",
          "media_results": {
            "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAAR0OCmYKl8AACgACHQ4LFKHWIcQAAA==",
            "result": {
              "__typename": "ApiMedia",
              "id": "QXBpTWVkaWE6DAABCgABHQ4KZgqXwAAKAAIdDgsUodYhxAAA",
              "media_key": "3_2093622310154518528"
            }
          },
          "media_url_https": "https://pbs.twimg.com/media/HQ4KZgqXwAATgCO.png",
          "original_info": {
            "focus_rects": [
              {
                "h": 1184,
                "w": 2115,
                "x": 0,
                "y": 371
              },
              {
                "h": 1555,
                "w": 1555,
                "x": 0,
                "y": 0
              },
              {
                "h": 1555,
                "w": 1364,
                "x": 0,
                "y": 0
              },
              {
                "h": 1555,
                "w": 778,
                "x": 0,
                "y": 0
              },
              {
                "h": 1555,
                "w": 2115,
                "x": 0,
                "y": 0
              }
            ],
            "height": 1555,
            "width": 2115
          },
          "sizes": {
            "large": {
              "h": 1506,
              "w": 2048
            }
          },
          "type": "photo",
          "url": "https://t.co/51qIY0oKZd"
        }
      ]
    },
    "card": null,
    "place": {},
    "entities": {},
    "quoted_tweet": null,
    "retweeted_tweet": null,
    "isLimitedReply": false,
    "communityInfo": null,
    "article": null
  },
  "retweeted_tweet": null,
  "isLimitedReply": false,
  "communityInfo": null,
  "article": null
}