TimeDistributedDense64

TimeDistributedDense64 is a neural network layer designed for processing sequence data. It applies a dense transformation with 64 output units to each timestep of the input independently, preserving the time structure of the sequence. For an input tensor with shape (batch_size, timesteps, input_dim), the output has shape (batch_size, timesteps, 64). Conceptually, it is equivalent to wrapping a Dense(64) layer with a TimeDistributed wrapper, so the same weights are used across all timesteps.

Parameters and computation

The layer uses a weight matrix of shape (input_dim, 64) and a bias vector of length 64. Because the transformation is shared across timesteps, the total number of trainable parameters is input_dim * 64 + 64, independent of the number of timesteps. Computationally, the layer performs a matrix multiplication for each timestep, resulting in a per-timestep feature expansion from input_dim to 64.

Activation and variants

By default, the layer can be linear, but an activation function (such as ReLU, tanh, or sigmoid) can be applied to the 64 outputs. The time-distributed structure means the same activation is applied every timestep. The layer can be combined with masking to handle variable-length sequences and with other layers such as recurrent networks or attention mechanisms.

Use cases

TimeDistributedDense64 is commonly used to transform per-timestep features before feeding them into recurrent layers, sequence-to-sequence models, or attention-based architectures. It serves as a lightweight, fixed-size per-timestep feature extractor, enabling downstream temporal modeling without altering the sequence length.

Relation to related concepts

It is a specific instance of the TimeDistributed layer concept and relates to Dense layers and recurrent neural networks. It is especially useful when a uniform per-timestep representation is desired before subsequent temporal processing.