Articles → MACHINE LEARNING → Information Content (Or Surprisal)

Information Content (Or Surprisal)

This article explains Information Content (or Surprisal).

Purpose

Information content, or surprisal, is a measure of how surprising an event is. Rare events carry more information than common ones.

Where P(x) is the probability of an event.

Event	Probability	Surprisal
Coin toss = Heads	0.5	-log2(0.5) = 1 bit
Dice roll = 6	1/6 ≈ 0.167	≈ 2.58 bits

Query/Feedback