Oozie is a workflow scheduler system that manages Hadoop work. Oozie will be discussed in the next blogs. First, we will examine the basic concepts, architecture and background of Oozie.
Oozie background
At the early stages of big data projects, data source is limited and so MapReduce is a good fit since it focuses on one task. However, as the project accelerates and the size of data gets bigger and bigger, MapReduce needs to connect to each other. Users need to manage the initial point and end point and schedule service for branch pattern. That’s why Oozie was created.
The basic concept of Oozie
Oozie is a workflow engine implemented from a Java servlet-container as a web application. The workflow is a group of several jobs including work through script languages such as MapReduce from Hadoop and Pig.
The workflow consists of two parts.
- Control node
- It assigns both the start point and end point and controls the path of work processing.
- There are start/end nodes that assign the start point and end point of work.
- There are decision/fork/join nodes which define various conditions to control the path of processing the work.
- Action node
- One workflow to perform operations using Hadoop MapReduce. Workflow relates to file system such as HDFS, Pig, SSH, HTTP, Email and Oozie sub workflow. Since Oozie can be extended, it is possible to apply extra actions.
Oozie manages workflow based on the DAG (Direct Acyclic Graph) rule. DAG has predecessors and successors and it does not include the cycle within graph to make the order of operation effective.
Oozie assigns frequency information when some work keeps being processed and it sets the order of work based on the DAG rule. After user sets job dependency for each work, Oozie manages to process by order using job dependency information.
Oozie Architecture
Oozie consists of two main parts.
- Workflow engin
- It provides basic functions to define workflow or help real-operation following defined workflow.
- Coordinator engine
- Almost all work in Oozie is scheduled to be processed at a special time or it will start after checking normal processing through predecessor. In a special case, it performs successive work to follow upn special events which happens outside of Oozie.
- The coordinator system of Oozie helps to proceed as planned following assigned conditions. This system helps users to express conditions to process special work in programming language. (Usually called as trigger).
Common questions
Is Apache Oozie still maintained?
Oozie remains an Apache project and is still present in many existing Hadoop and Cloudera deployments. Check the Apache Oozie project site directly for the current release status before planning new work around it.
What is the difference between a workflow and a coordinator in Oozie?
A workflow is a single run of a DAG of actions from start to end. A coordinator schedules workflow instances repeatedly, based on time, data availability, or both.
Can Oozie run Spark or Impala jobs?
Oozie has a native Spark action type. It has no native Impala action, so Impala queries are usually run through a shell action that calls impala-shell, with Oozie handling the surrounding sequencing.
Does Oozie replace Hadoop's scheduler?
No. Oozie orchestrates jobs at a higher level and submits them to the cluster's resource manager (YARN), which still handles the low-level scheduling of containers and resources.
How does Oozie compare to Apache Airflow?
Both schedule and sequence jobs, but Oozie is built specifically around Hadoop job types and XML definitions, while Airflow defines pipelines in general-purpose code and is not tied to Hadoop. Teams already running Hadoop often keep Oozie for existing pipelines and use a general-purpose orchestrator for new ones.
Sources
About this page
Last reviewed October 30, 2026 by the Nine Bits editors. Read our editorial policy or report an error.