@lennysan
I asked @echen why Claude writes (and codes) so much better than other models. His answer: higher-quality training data. "Most people don't understand what quality even means in this space. They think you could just throw bodies at a problem and get good data, and that's completely wrong. Let me give you an example. Imagine you wanted to train a model to write an eight-line poem about the moon. What makes it a good poem? If you don't think deeply about quality, you'll be like, is this a poem? Does it contain eight lines? Does it contain the word moon? You check all these boxes? So then yeah, sure, you say it's a great poem. But that's completely different from what we want. We are looking for Nobel Prize-winning poetry. Is this poetry unique? Is it full of subtle imagery? Does it surprise you, and tug at your heart? Does it teach you something about the nature of moonlight? Does it play through emotions, and does it make you think? That's what we are thinking about when we think about a high-quality poem."