DeiTSmall
DeiTSmall is a lightweight implementation of the Data-efficient Image Transformer (DeiT) model. DeiT itself is a vision transformer designed to achieve high performance with significantly less training data compared to earlier transformer architectures. DeiTSmall is a smaller version of the original DeiT, meaning it has fewer parameters and a reduced computational footprint. This makes it more suitable for applications where computational resources are limited, such as mobile devices or embedded systems, or for faster experimentation and prototyping.
The core innovation of DeiT lies in its training strategy, which incorporates a data-efficient training methodology.