@kmelve
sat down and actually read @rosmine's research paper on this. https://t.co/vxqTirXxrS first of all, i was guilty of having a "take" based on this post, pointing out how the copy in the screenshot is not a great example of what "good writing" is, despite scoring 100 on "human written" in @pangram. I recommend actually reading the work before commenting on it - because sloppy takes are as lazy as sloppy texts. (shame on me for joining the band wagon) but sleeping on it, i am grateful that @rosmine took the time to dive into this stuff. AI-slop fatigue is real, and we should support efforts to make agents produce communication that is clear and lucid, and doesn't feel overly synthetic but there are some assumptions in this, however, that is interesting to question when it comes to "what makes for good writing" as far as i understand, Deft is trained to have more diversity/variation in tokens, so less repetition of what we are recognizing as "AI-tells" (certain word and stylistic choices). And i totally agree with @rosmine that: "LLMs are not the cause of slop. Lack of effort/care is. If you spend days researching and planning a blog post, and put all the information into a detailed, well-structured outline, and ask ChatGPT to generate the post based on the outline, then the output will be interesting to read, even if the text has a lot of em-dashes." I argued the same in our eng blog announcement post yesterday: https://t.co/sV3cEP4S9G BUT! I still feel that this report (at least somewhat), but especially the various takes on it, conflates something sounding "human" with it being "good." Tricking @pangram doesn't make a text well written. Some reflections: - Making writing sound more "human" by means of adding more variation in word/style choices, doesn't make it better - What makes for a "good" text is highly contextual. If you are writing a recipe or instructions, repetition and stylistic stringency is highly desirable - AI-tells cuts deeper than just word choices, observant readers will start to be sensitive to the lack of certain devices, ways of arguing, structure, etc. - Interesting writing often comes from doing synthesis of unexpected things in a way that bring clarity. I have still to see that from LLMs as we usually interact with them (but it's probably possible to get them to do this). with that being said - i'm grateful that this research was shared with the wider community, and will applaud any effort to make the user experience of interacting with agents, and the stuff we make with agents, better. 🫡