Shannoninformációtartalom

Shannoninformációtartalom, often translated as Shannon information content or surprisal, is a concept introduced by Claude Shannon in his seminal work on information theory. It quantifies the amount of information contained in a single event or outcome of a random variable. The information content of an event is inversely proportional to its probability; a less probable event carries more information.

Mathematically, the Shannon information content of an event $x$ with probability $P(x)$ is defined as $I(x) = -\log_b P(x)$. The base of the logarithm, $b$, determines the unit of information. When $b=2$, the unit is bits. When $b=e$ (natural logarithm), the unit is nats. When $b=10$, the unit is dits or bans. The negative sign ensures that information content is always non-negative, as probabilities are between 0 and 1.

The core idea is that the more surprising an event is, the more we learn from its occurrence. For instance, if an event has a probability of 1 (certainty), its information content is 0, meaning we gain no new information from its occurrence. Conversely, an event with a very small probability, like winning the lottery, carries a high information content. Shannoninformációtartalom is fundamental to understanding data compression, communication channel capacity, and various statistical modeling techniques. It provides a probabilistic measure of how much uncertainty is resolved by observing a particular outcome.

---