@wandb
Shared prefix fixes this. Instead of 16 separate sequences, you lay the tokens out as a tree. The system prompt gets encoded once and the 16 rollout branches hang off it. The mask lets every branch see the shared prefix while staying blind to the other branches. https://t.co/pIqAzoaHAe