Generalization Bounds for Autoregressive Processes and In-Context Learning
2025
Rephrasability
A property of a sequential data-generating process that relates prediction on a new context to denoising problems over contexts generated by the process.
A property of a sequential data-generating process that relates prediction on a new context to denoising problems over contexts generated by the process.
Combined with an appropriately stable predictor class, rephrasability provides a route from in-domain risk control to out-of-domain prediction guarantees.