Incremental Learning of Sparse Attention Patterns in Transformers
2026
Implicit bias
The study of how a learning algorithm selects among multiple solutions even without an explicit preference in the objective.
The study of how a learning algorithm selects among multiple solutions even without an explicit preference in the objective.
The selected solution can depend on the algorithm, initialization, parameterization, and training trajectory.