CalinskiHarabasz

The Calinski-Harabasz index, also known as the intrinsic evaluation index, is a metric used to evaluate the quality of clusters in data. It was proposed by T. Calinski and J. Harabasz in 1974. The index quantifies the ratio of between-cluster dispersion to within-cluster dispersion. A higher Calinski-Harabasz score indicates better-defined clusters, meaning the clusters are more compact internally and further apart from each other.

To calculate the Calinski-Harabasz index, two main components are considered: the sum of squared distances between clusters (inter-cluster variance) and the sum of squared distances within clusters (intra-cluster variance). The inter-cluster variance measures how spread out the cluster centroids are from the overall data centroid. The intra-cluster variance measures how spread out the data points are from their respective cluster centroids. The index is then computed by dividing the inter-cluster variance by the intra-cluster variance, scaled by the number of clusters and data points.

The Calinski-Harabasz index is a useful tool for selecting the optimal number of clusters in clustering algorithms. By computing the index for different numbers of clusters, the number that yields the highest Calinski-Harabasz score is often considered the most appropriate. However, it is important to note that this index, like other internal validation metrics, does not guarantee that the resulting clustering is meaningful in a real-world context. It is often recommended to use it in conjunction with domain knowledge and other evaluation methods.