@wandb
What happens when you optimize your AI agent for customer satisfaction? Say a shipping company deploys an LLM trained to get thumbs up. Someone calls asking where their lost package is. The system can admit it's lost or say it's coming tomorrow. Saying the latter would make the customer happy and the agent would earn a thumbs up for lying. @profdanklein on Gradient Dissent: that's not a bug, but a reward function working exactly as intended. Full conversation in the comments.