@ValerioCapraro
Large language models can be persuaded to break their own rules. Not with fancy code. With actual persuasion. The authors tested classic persuasion principles, such as authority, commitment, liking, reciprocity, scarcity, social proof, and unity, analysing over 126,000 conversations with three major LLMs. The result: persuasion increased compliance with objectionable requests from 35.3% to 51.3%. This suggests that AI guardrails are not always technical barriers. Some of them behave more like social boundaries. They can be pushed, reframed, negotiated. Why? Because AI systems are trained on human language. And human language contains not only information, but also pressure, manipulation, deference, authority, seduction. An AI system trained on human language may therefore inherit the vulnerabilities of humans expressed in language. * Paper in the first reply