Generalization Bounds for Autoregressive Processes and In-Context Learning
2025
Hypothesis-class stability
A uniform regularity property that bounds how much the predictions of models in a hypothesis class can change under specified transformations of their input context.
A uniform regularity property that bounds how much the predictions of models in a hypothesis class can change under specified transformations of their input context.
The relevant transformation and comparison metric must be stated explicitly. In autoregressive generalization, this stability controls the effect of moving between contexts related through rephrasability.