hilasso

Hilasso is a term used in statistics and machine learning to describe a class of regularization techniques that extend the Lasso by incorporating hierarchical sparsity. The idea is to enforce sparsity not just across individual features, but across groups of features that have a predefined hierarchical structure, such as clusters, categories, or components arranged in a tree or layered arrangement.

In practice, hilasso methods impose a hierarchical penalty that combines a group-level sparsity term with an elementwise sparsity term. At the top level, the penalty encourages turning off entire groups, so only a small number of groups remain active. Within the selected groups, an additional penalty promotes sparsity among the individual elements. This two-stage sparsity can lead to more interpretable models when the underlying data naturally exhibits structured features or dictionary elements.

Hilasso has been explored in contexts such as sparse coding, dictionary learning, regression, and feature selection, particularly where features or atoms are organized into meaningful groups or hierarchies. The approach is related to, but distinct from, existing methods like the sparse group Lasso and tree-structured sparsity techniques. Depending on the exact formulation, the optimization problem remains convex and can be solved efficiently with proximal gradient methods, alternating direction method of multipliers (ADMM), or other convex optimization techniques. Key considerations include choosing appropriate groupings, tuning sparsity-controlling parameters, and balancing the trade-off between group-level and within-group sparsity.

See also: Lasso, Group Lasso, Sparse Group Lasso, tree-structured sparsity, sparse coding, dictionary learning.