[TAJO-540] (Umbrella) Implement Tajo Query Scheduler

• Type: New Feature
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Hyunsik Choi
• Created: 21/Jan/14 07:17
• Updated: 21/Oct/15 04:39

Description

Currently, there is no Tajo query scheduler. So, all queries launched simultaneously compete cluster resource which is managed by TajoResourceManager.

In this issue, we will investigate, design, and implement a Tajo query scheduler. This is an umbrella issue for that. We will create subtasks for them.

Issue Links

• relates to: TAJO-550 TajoMaster chooses mostly the same query master
• relates to: TAJO-773 (Umbrella) Integrate with Apache Hadoop YARN as resource allocator
• relates to: TAJO-855 Refactor TaskRunnerManager/TaskRunner/Task
• relates to: TAJO-703 Sparrow-like scheduler
• relates to: TAJO-323 Refactor inconsistent class and variable names
• requires: TAJO-1397 Resource allocation should be fine grained.
• requires: TAJO-1255 Remove LazyTaskScheduler and its related things

Sub-Tasks

• WorkerResourceManager should be broke down into 3 parts: Resolved
• Move container allocation from SubQuery down to QueryUnitAttempt: Resolved
• Remove yarn-related code from tajo-core: Resolved
• Improve worker resource and Add simulator: Resolved
• Implement TaskManager: Resolved