HiveQL

HiveQL, or Hive Query Language, is the SQL-like query language used with Apache Hive to read, write, and manage data stored in the Hadoop ecosystem, typically in HDFS or compatible storage. HiveQL queries are translated by the Hive engine into MapReduce, Tez, or Spark jobs and executed across a Hadoop cluster. The language is designed for data warehousing and analytics on large-scale datasets and emphasizes batch processing over low-latency interaction.

Features and syntax: HiveQL provides familiar SQL constructs such as SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY, and JOIN, as well as DDL statements like CREATE TABLE and ALTER. It supports partitioning and bucketing, and uses a metastore for schema and metadata management. Data types include primitive types (int, bigint, double, boolean, string, timestamp, date) and complex types (ARRAY, MAP, STRUCT). Storage formats supported include TEXTFILE, ORC, Parquet, and Avro. Hive supports user-defined functions (UDFs) and, in recent versions, vectorized execution and ACID transactions under certain conditions.

Execution and performance: Queries are compiled into underlying execution engines such as MapReduce, Tez, or Spark. LLAP provides in-memory acceleration for interactive workloads. Performance is enhanced by partitioning, bucketing, and columnar formats, along with various optimization features and planning options in newer releases.

Limitations and scope: HiveQL is not fully ANSI SQL compliant and is optimized for batch analytics rather than real-time querying. It relies on data being stored in compatible formats and on accurate metastore schemas, which influences latency and consistency characteristics for large-scale scans.

See also: Apache Hive, Hadoop, SQL-on-Hadoop ecosystems.