[TEZ-2834] Make Tez preemption resilient to incorrect free resource reported by YARN - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.7.1
Fix Version/s: 0.7.1, 0.5.5, 0.6.3, 0.8.1-alpha
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed

Description

Will attach the DAG.

Repro for reference: TPC-DS q_70 @ 30 TB scale.

"Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched slightly late. But before "Reducer 9" can get scheduled, slots are taken up by "Map 1", which is not preempted for running "Reducer 9".

This is with 0.7.1 codebase.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

TEZ-2834.2.patch
19/Sep/15 20:48
25 kB
Bikas Saha
TEZ-2834.1.patch
18/Sep/15 23:33
11 kB
Bikas Saha
hive_view.png
16/Sep/15 09:35
62 kB
Rajesh Balamohan
DAG_view.png
16/Sep/15 09:35
103 kB
Rajesh Balamohan
application_1442254312093_0095.2.log.gz
16/Sep/15 09:35
7.43 MB
Rajesh Balamohan
application_1442254312093_0095.1.log.gz
16/Sep/15 09:35
6.68 MB
Rajesh Balamohan

Activity

People

Assignee:: Bikas Saha

Reporter:: Rajesh Balamohan

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 16/Sep/15 09:27

Updated:: 12/Oct/15 20:46

Resolved:: 21/Sep/15 19:22