Cross-entropy loss

A loss that scores a predicted probability distribution by the negative log-probability assigned to the observed outcome.

A loss that scores a predicted probability distribution by the negative log-probability assigned to the observed outcome.

is aObjective All entries Open in graph

For conditional language modeling, it is applied token by token to the distribution predicted from the preceding context.

Connections

related to
Next-token prediction
specialization of
Loss function

Related publications