{
  "type": "tweet",
  "id": "2029701092347630069",
  "url": "https://x.com/karpathy/status/2029701092347630069",
  "twitterUrl": "https://twitter.com/karpathy/status/2029701092347630069",
  "text": "nanochat now trains GPT-2 capability model in just 2 hours on a single 8XH100 node (down from ~3 hours 1 month ago). Getting a lot closer to ~interactive! A bunch of tuning and features (fp8) went in but the biggest difference was a switch of the dataset from FineWeb-edu to NVIDIA ClimbMix (nice work NVIDIA!). I had tried Olmo, FineWeb, DCLM which all led to regressions, ClimbMix worked really well out of the box (to the point that I am slightly suspicious about about goodharting, though reading the paper it seems ~ok).\n\nIn other news, after trying a few approaches for how to set things up, I now have AI Agents iterating on nanochat automatically, so I'll just leave this running for a while, go relax a bit and enjoy the feeling of post-agi :). Visualized here as an example: 110 changes made over the last ~12 hours, bringing the validation loss so far from 0.862415 down to 0.858039 for a d12 model, at no cost to wall clock time. The agent works on a feature branch, tries out ideas, merges them when they work and iterates. Amusingly, over the last ~2 weeks I almost feel like I've iterated more on the \"meta-setup\" where I optimize and tune the agent flows even more than the nanochat repo directly.",
  "source": "Twitter for iPhone",
  "retweetCount": 351,
  "replyCount": 221,
  "likeCount": 4263,
  "quoteCount": 58,
  "viewCount": 281926,
  "createdAt": "Thu Mar 05 23:30:25 +0000 2026",
  "lang": "en",
  "bookmarkCount": 1498,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "2029701092347630069",
  "displayTextRange": [
    0,
    274
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "karpathy",
    "url": "https://x.com/karpathy",
    "twitterUrl": "https://twitter.com/karpathy",
    "id": "33836629",
    "name": "Andrej Karpathy",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1296667294148382721/9Pr6XrPB_normal.jpg",
    "coverPicture": "https://pbs.twimg.com/profile_banners/33836629/1407117611",
    "description": "I like to train large deep neural nets. Previously Director of AI @ Tesla, founding team @ OpenAI, PhD @ Stanford.",
    "location": "Stanford",
    "followers": 1886573,
    "following": 1055,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Tue Apr 21 06:49:15 +0000 2009",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {
        "urls": [
          {
            "display_url": "karpathy.ai",
            "expanded_url": "https://karpathy.ai",
            "indices": [
              0,
              23
            ],
            "url": "https://t.co/0EcFthjJXM"
          }
        ]
      }
    },
    "fastFollowersCount": 0,
    "favouritesCount": 22216,
    "hasCustomTimelines": true,
    "isTranslator": false,
    "mediaCount": 853,
    "statusesCount": 9987,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [
      "1617979122625712128"
    ],
    "profile_bio": {},
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "allow_download_status": {
          "allow_download": true
        },
        "display_url": "pic.x.com/qragh6hJL6",
        "expanded_url": "https://x.com/karpathy/status/2029701092347630069/photo/1",
        "ext_media_availability": {
          "status": "Available"
        },
        "features": {
          "large": {
            "faces": [
              {
                "h": 117,
                "w": 117,
                "x": 39,
                "y": 547
              }
            ]
          },
          "medium": {
            "faces": [
              {
                "h": 69,
                "w": 69,
                "x": 23,
                "y": 320
              }
            ]
          },
          "orig": {
            "faces": [
              {
                "h": 185,
                "w": 185,
                "x": 62,
                "y": 859
              }
            ]
          },
          "small": {
            "faces": [
              {
                "h": 39,
                "w": 39,
                "x": 13,
                "y": 181
              }
            ]
          }
        },
        "id_str": "2029699270820122624",
        "indices": [
          275,
          298
        ],
        "media_key": "3_2029699270820122624",
        "media_results": {
          "result": {
            "media_key": "3_2029699270820122624"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/HCrwu6YaUAAoAlh.jpg",
        "original_info": {
          "focus_rects": [
            {
              "h": 1799,
              "w": 3212,
              "x": 0,
              "y": 215
            },
            {
              "h": 2014,
              "w": 2014,
              "x": 35,
              "y": 0
            },
            {
              "h": 2014,
              "w": 1767,
              "x": 159,
              "y": 0
            },
            {
              "h": 2014,
              "w": 1007,
              "x": 539,
              "y": 0
            },
            {
              "h": 2014,
              "w": 3212,
              "x": 0,
              "y": 0
            }
          ],
          "height": 2014,
          "width": 3212
        },
        "sizes": {
          "large": {
            "h": 1284,
            "resize": "fit",
            "w": 2048
          },
          "medium": {
            "h": 752,
            "resize": "fit",
            "w": 1200
          },
          "small": {
            "h": 426,
            "resize": "fit",
            "w": 680
          },
          "thumb": {
            "h": 150,
            "resize": "crop",
            "w": 150
          }
        },
        "type": "photo",
        "url": "https://t.co/qragh6hJL6"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {
    "hashtags": [],
    "symbols": [],
    "urls": [],
    "user_mentions": []
  },
  "quoted_tweet": null,
  "retweeted_tweet": null,
  "article": null
}