@karpathy
@andrew_n_carr Yeah, $10B is the difference in finding it first and ~5 years ago. :) I just love reproducing landmark results for much cheaper, it's so fun! Reproducing LeCun 1989 was super fun too: https://t.co/oOZcQW3Y9H What runs unoptimized on a consumer laptop in 1 minute was a state of the art neural net trained for days in 1989. Another favorite example: CIFAR-10. In 2011 state of the art was 77%. I estimated human accuracy to be ~94% but said that performance might go up to 85-90%. https://t.co/KJl0V4T0ei Now you can speedrun to 94% accuracy in 1.98 seconds on a single GPU (yes, <2 seconds). https://t.co/wHUQs6htdV So e.g. right now GPT-2 (imo the landmark result that launched LLMs and where the modern stack is basically in full form) is ~$500, but I'm unreasonably obsessed with how much that can be brought down.