döntésfaalgoritmusoknál

Decision tree algorithms are a class of supervised learning methods used for classification and regression. They predict a target variable by learning a set of decision rules inferred from data. A tree consists of internal decision nodes, branches, and leaf nodes that contain predictions. Each internal node tests a feature value; traversal from the root to a leaf yields a prediction.

Construction: Most algorithms use greedy top-down induction. At each step, the algorithm selects the feature and threshold that splits the data to maximize a purity gain, such as information gain (entropy) or Gini impurity for classification, or minimize variance for regression. The process repeats recursively on resulting subsets until stopping criteria are met (minimum samples per leaf, maximum depth, or impurity threshold).

Variants: CART (Classification and Regression Trees) handles both classification and regression and uses the Gini impurity for classification and variance reduction for regression. ID3 and its successor C4.5 focus on categorical features; C4.5 extends ID3 to handle continuous features and missing values and performs pruning.

Training and pruning: Pruning reduces overfitting by removing branches that do not provide power on unseen data. Methods include cost-complexity pruning and cross-validation guided pruning.

Strengths and limitations: Decision trees are interpretable and fast at inference, handle mixed data types, and require little data preparation. They are prone to overfitting without pruning, can be unstable to small data changes, and may be biased toward features with many distinct values or classes.

Applications: broad use in medicine, finance, policy and marketing for classification and regression tasks, such as diagnosing diseases, credit scoring, and estimating prices.

History: Early work includes CART by Breiman et al. (1984), ID3 by Quinlan (1986), and C4.5 as its successor (1993).