[TEZ-1665] DAGScheduler should provide a priority range instead of an exact priority

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.2
• Hadoop Flags: Reviewed
• Assignee: Bikas Saha
• Reporter: Bikas Saha
• Created: 14/Oct/14 20:38
• Updated: 08/Nov/14 20:25
• Resolved: 15/Oct/14 22:19

Description

Specifying the exact priority leaks task attempt concepts into the DAGScheduler. Eg. the DAGScheduler increases the priority of attempt-re-executions arbitrarily. Providing a range allows these decisions to be made at the task attempt level. e.g. when there are no competing tasks then there is no reason to increase the priority. Similarly for speculation, the speculative task priority could be lower than normal tasks because we may want the first attempt of other tasks to run ahead of a speculative task. These relative priorities within a vertex can be made locally instead of leaking into the DAGScheduler.

Issue Links

• is depended upon by: TEZ-14 Support MR like speculation capabilities based on latency deviation from the mean