Databasehenting

Databasehenting refers to the process of retrieving data stored in databases for various purposes, including data migration, integration, analysis or redistribution. The term is used in Norwegian IT contexts and can apply to both internal data retrieval within an organization and the collection of data from external sources, subject to legal access rights.

Common methods include querying databases with structured query language (SQL), using application programming interfaces (APIs) to fetch data, exporting data from database management systems, and building data pipelines that extract, transform and load data into a data warehouse or data lake. In enterprise settings, database harvesting is often part of ETL/ELT workflows and master data management.

Use cases range from moving data between systems during migrations, consolidating data for analytics, feeding business intelligence, to supporting research and compliance reporting. Data harvesting can involve both structured data and semi-structured data such as JSON or XML, depending on source formats and access methods.

Legal and ethical considerations are central. Access rights, licenses, and terms of service govern permissible harvesting. In the European Economic Area, privacy laws such as the GDPR regulate handling personal data, requiring lawful basis, purpose limitation, minimization, and, when appropriate, anonymization or pseudonymization.

Challenges include ensuring data quality and consistency, keeping schemas and metadata up to date, managing performance impact on source systems, and securing credentials and data in transit. Unauthorized harvesting or scraping can be illegal and lead to penalties; organizations should implement governance, auditing, and safeguards.