Andmekomplekti

A data set (andmekomplekti) is a structured collection of data records used for analysis, modeling, or reporting. In machine learning, a data set typically consists of samples (rows) and features (columns), with a possible target variable for supervised tasks.

Components include features describing attributes, samples representing instances, and, in supervised contexts, a label column. Metadata describes provenance, license, date of creation, data quality notes, and data types.

Formats and storage: Datasets are stored in formats such as CSV, JSON, Parquet, or within databases. They may be organized in folders with documentation, versioned, and used in pipelines.

Quality and preprocessing: Datasets may contain missing values, inconsistent entries, or outliers. Preprocessing steps include cleaning, normalization, encoding, and splitting into training, validation, and test sets. Data quality affects model performance and reproducibility.

Usage and ethics: Datasets are used for data analysis, ML model training, and benchmarking. Ethical and legal considerations include privacy, consent, de-identification, and compliance with data protection laws (e.g., GDPR). Licensing determines reuse rights.

Examples and limitations: Public datasets support research and education but can introduce bias if not representative. Proper documentation and provenance improve transparency and reproducibility.