TextLaspeyres

TextLaspeyres is a metric used to quantify changes in the composition of words in texts over time or across documents. Inspired by the Laspeyres price index from economics, it uses a fixed basket of terms derived from a base period or document set. By focusing on the distribution of words rather than raw length, textLaspeyres aims to capture shifts in topics, style, or vocabulary.

In its typical form, let f0_i be the base-period relative frequency of word i, and f1_i the current relative frequency. The textLaspeyres index I is defined as I = sum_i f0_i * f1_i divided by sum_i f0_i^2. The index equals 1 when the current distribution matches the base distribution. Values greater than 1 indicate convergence toward high-weight base terms, while values below 1 indicate divergence away from them. The base weights f0_i reflect the importance of each term in the fixed basket and can be adjusted for domain or time period.

The method yields a scalar measure that is easy to interpret and decomposable by term: the contribution of word i is proportional to f0_i*(f1_i - f0_i). It allows researchers to identify which words drive change and to compare multiple texts on a common footing, even if they vary in length.

Limitations include sensitivity to the choice of base basket, potential instability with rare terms, and the need for careful normalization. It is not a likelihood or distance metric and should be used alongside other text analysis tools.

See also: Laspeyres price index, price indices, corpus linguistics, topic modeling, text similarity.