Informaatiopitoisuus

Informaatiopitoisuus, often translated as information content or information value, is a concept used to quantify the amount of information conveyed by a particular event or message. In essence, it measures how surprising or unexpected an event is. A highly improbable event carries more information than a very likely one, because its occurrence provides a significant update to our knowledge.

The concept is formally defined in information theory. The information content of an event with probability P is given by the negative logarithm of its probability, typically expressed as -log₂(P). The base-2 logarithm is commonly used, meaning the information content is measured in bits. If an event has a probability of 1, its information content is 0 bits, as it provides no new information. Conversely, an event with a probability of 0.5 has an information content of 1 bit, as its occurrence halves the uncertainty.

Informaatiopitoisuus is a fundamental concept in understanding data compression, communication systems, and statistical inference. It helps in designing efficient ways to encode and transmit information and in assessing the significance of observed data. For example, in natural language processing, rare words or grammatical structures tend to have higher information content than common ones. The more unexpected a piece of information is, the more it contributes to reducing uncertainty and, therefore, the more "information" it contains.