Apache Hive: How It Works and When to Use It

A hive is data warehouse infra tool to be used for querying, analyzing and processing structured data on Hadoop. We are going to introduce Hive over several times. First of all, we look into the feature, Architecture and configuration.

Feature

• A schema is stored on Database and processed data is stored on HDFS.
• It is made for OLAP
• It offers a query language in type of SQL called as “HiveQL”

• It is so fast and easy to handle and expend.

Architecture

• User Interface
  • A Hive is data warehouse infra software to interact between users and HDFS. User interfaces supported by Hive are Hive Web UI, Hive command line and Hive HD Insight (Window server).
• Meta Store
  • Hive selects each data server and saves a table, database, row of table, data format and HDFS mapping.
• HiveQL Process Engine
  • A HiveQL is language to query schema information of Metastore similar to SQL language. Instead of writing MapReduce program by Java language, it is able to write query to MapReduce working.
• Execution Engine
  • HiveQL Execution Engine is a part of combination between HiveQL process engine and MapReduce. The execution engine processes query and generates the same result with MapReduce.
• HDFS or HBASE
  • A Hadoop distribution file system or HBASE is date storing technology saving data on file system.

Processing step

Step 1 Execute Query

• Hive interfaces such as Command Line or Web UI send a query to driver (Database driver like JDBC,ODBC) and processes it.

Step 2 Get Plan

• The driver checks up a query, plan and requirement with assistance from a query compiler.

Step 3 Get Metadata

• The compiler sends a MetaData request to Metastore.

Step 4 Send Metadata

• Metastore sends the metadata as response for the compiler.

Step 5 Send Plan

• The compiler checks up requirement and sends a plan to the drive again.

Step 6 Execute Plan

• The driver sends the plan to an execution engine.

Step 7 Execute job

• In internally, a process of execution working is the MapReduce. The execution engine transfers executive working to JobTracker in name nodes and assigns the working for a TaskTackerin in a data nodes. The query executes MapReduce working in here.

Step 7.1 Metadata Ops

• During the processing, the execution engine can execute the Metadata working with the Metastore.

Step 8 Fetch Result

• The executive engine receives the result from data nodes.

Step 9 Send Results

• The executive engine sends a result value to the driver.

Step 10 Send Results

• The driver sends the result to the Hive interface.