SparseCategoricalCrossentropy

SparseCategoricalCrossentropy is a loss function used for multi-class classification when ground-truth labels are provided as integers rather than one-hot vectors. It is implemented in TensorFlow Keras as tf.keras.losses.SparseCategoricalCrossentropy and is compatible with models that output either class probabilities or raw logits. The typical inputs are y_true with shape (batch_size,) containing integers in the range [0, num_classes-1], and y_pred with shape (batch_size, num_classes).

The loss evaluates the cross-entropy between the true class and the predicted distribution. If from_logits is set to True, the function expects unnormalized logit scores and applies a numerically stable softmax internally before computing the negative log-likelihood. If from_logits is False, y_pred should represent a probability distribution (for example, the output of a softmax), and the loss computes the standard cross-entropy with those probabilities. The loss supports sample weighting and various reduction methods, and it raises an error if any label falls outside the valid class index range.

A key distinction from CategoricalCrossentropy is that SparseCategoricalCrossentropy uses integer labels directly, avoiding the need to one-hot encode y_true. This can be more memory-efficient for problems with many classes. It is common to set from_logits=True when using a model that outputs raw logits, as this tends to be more numerically stable than applying softmax early in the pipeline.

In practice, users specify the loss in model compilation, e.g., loss=SparseCategoricalCrossentropy(from_logits=True), and ensure that the model’s final layer outputs either logits or probabilities consistent with the chosen setting.