Competitive-to-cooperative attention dynamics
Produced by Incremental Learning of Sparse Attention Patterns in Transformers
Training-trajectory model
A formal or empirical model that describes how a learning system changes over the course of training.
A formal or empirical model that describes how a learning system changes over the course of training.
It may characterize parameter updates, representation growth, or the emergence of particular computations.