VLM

VLM most commonly stands for Visual Language Model, a class of artificial intelligence models that integrate visual perception with natural language processing to understand and generate information across image and text modalities.

A Visual Language Model typically includes a visual encoder, such as a convolutional neural network or a vision transformer, that converts images into feature representations, and a language component, often a transformer-based text encoder or decoder, that processes and produces language. A cross-modal fusion mechanism enables the model to reason jointly over both modalities. Training uses large datasets of image–text pairs and objective functions that cover tasks like image captioning, visual question answering, and image–text retrieval. Some approaches employ contrastive learning to align image and text embeddings, while others train end-to-end on multiple tasks.

Applications span numerous domains, including generating captions for images, answering questions about visual content, enabling multimodal search, and improving accessibility by providing descriptive text for visually impaired users. In robotics and autonomous systems, visual language models can support instruction following and scene understanding.

Key challenges include effectively aligning representations across vision and language, data efficiency, biases present in training data, and safety considerations when generating or interpreting visual text. Evaluation typically relies on benchmarks for VQA accuracy, image captioning metrics such as BLEU and CIDEr, and retrieval metrics that measure cross-modal recall.

Beyond visual language models, the acronym VLM is also used in other contexts to denote different concepts, but Visual Language Model remains the primary reference in AI discussions.