Out-of-domain generalization from rephrasability and stability
Produced by Generalization Bounds for Autoregressive Processes and In-Context Learning
Next-token prediction
The task of predicting a conditional distribution for the next token given a preceding token context.
The task of predicting a conditional distribution for the next token given a preceding token context.
It is commonly trained with cross-entropy loss and evaluated through predictive risk or log loss.
Connections
Has applicable result
Sample-complexity bounds for next-token prediction
Produced by On the Sample Complexity of Next-Token Prediction