Promtail

Promtail is an agent designed to collect and ship logs to Grafana Loki. It is part of the Grafana Loki ecosystem and is commonly deployed alongside Loki to provide log data from local sources to the Loki index.

Promtail can read log data from several sources, including local log files, the systemd journal, and Docker container logs. It tails files and reads journal entries, converting them into log streams that Loki can index. It uses a configuration file promtail.yaml to define how logs are discovered and labeled. The configuration employs scrape_configs to describe targets and label transformations. Each scrape config can apply pipeline_stages to parse or transform lines (for example, multiline handling, JSON parsing, or regular expressions) and to attach metadata as labels.

Promtail tracks progress using a positions file so that on restart it resumes where it left off. It then pushes the collected log streams to Loki via the Loki push API. The labels on streams enable flexible querying in Grafana. Typical labels include job and instance, but users can define their own to reflect the source and context of the logs.

Promtail is implemented in Go and is open source. It is commonly deployed as a daemon or sidecar in a variety of environments, including Kubernetes as a DaemonSet or on individual hosts, to feed Loki with a scalable, label-rich stream of logs.