Out-of-domain generalization from rephrasability and stability
Produced by Generalization Bounds for Autoregressive Processes and In-Context Learning
Generalization
The ability of a learned model to perform well on data that was not used to fit its parameters.
The ability of a learned model to perform well on data that was not used to fit its parameters.
It separates learning a reusable pattern from merely reproducing the observations seen during training.
Connections
Characterized by
Sample-complexity bounds for next-token prediction
Produced by On the Sample Complexity of Next-Token Prediction
Related publications
Generalization Bounds for Autoregressive Processes and In-Context Learning
2025
Re Can gradient clipping mitigate label noise?
2021
Semantic Perturbations with Normalizing Flows for Improved Generalization
2021
Adversarial Training with Orthogonal Regularization
2020