QuelleEntropie

QuelleEntropie is a term occasionally used in discussions of information theory to denote the entropy of a source, that is, the average information content produced by a stochastic source per symbol. In standard German terminology, more common labels are Quellentropie or Entropie der Quelle; QuelleEntropie appears as a neologism or domain-specific designation in some texts and online discussions to emphasize the origin of the information.

Mathematically, for a discrete memoryless source with an alphabet X and symbol probabilities p(x), the source entropy is H(X) = - sum_x p(x) log2 p(x) bits per symbol. For a stationary stochastic process {Xn}, the entropy rate is h = lim_{n→∞} (1/n) H(X1, ..., Xn), provided the limit exists; for independent and identically distributed sources, h = H(X). In this sense, QuelleEntropie aligns with the concept of source entropy or entropy rate used in coding theory.

Applications of the concept include determining fundamental limits for lossless data compression via the source coding theorem: on average, no encoding can achieve a representation shorter than the source entropy per symbol. The notion is central to designing efficient codes and understanding the efficiency of compression schemes, especially when the source exhibits memory or structure across symbols.

Examples illustrate the idea: for a fair binary source, H = 1 bit per symbol; for a biased binary source with p(0)=0.9 and p(1)=0.1, H ≈ 0.469 bits per symbol. See also Entropy (information theory), Source coding theorem, and Entropy rate. Note that in formal texts the terms Quellentropie or Entropie der Quelle are preferred, while QuelleEntropie is a less common variant.