[TEZ-2251] Race condition in VertexImpl & Edge causes DAG to hang - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.7.0
Component/s: None
Labels:
None

Target Version/s:

0.7.0
Hadoop Flags:

Reviewed

Description

Scenario:

Run TPCH query20 (https://github.com/cartershanklin/hive-testbench/blob/master/sample-queries-tpch/tpch_query20.sql) at 1 TB scale (tez-master branch, hive trunk)
Enable auto reduce parallelism
DAG didn't complete and got stuck in "Reducer 6"

Vertex parallelism of "Reducer 5 & 6" happens within a span of 3 milliseconds, and tasks of "reducer 5" ends up producing wrong partition details as it sees the updated task numbers of reducer 6 when scheduled. This causes, job to hang.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

tez_2251_dag.png
30/Mar/15 12:23
107 kB
Rajesh Balamohan
hive_console.png
30/Mar/15 12:23
80 kB
Rajesh Balamohan
TEZ-2251.VertexImpl.patch
01/Apr/15 23:24
7 kB
Rajesh Balamohan
TEZ-2251.fix_but_slows_down.patch
01/Apr/15 23:24
2 kB
Rajesh Balamohan
tez-2251.vertexpatch.am.log.gz
02/Apr/15 00:55
1.38 MB
Rajesh Balamohan
TEZ-2251.VertexImpl.readlock.patch
02/Apr/15 04:28
4 kB
Rajesh Balamohan
TEZ-2251.2.patch
02/Apr/15 05:59
3 kB
Rajesh Balamohan
TEZ-2251.3.patch
02/Apr/15 23:14
3 kB
Rajesh Balamohan

Activity

People

Assignee:: Rajesh Balamohan

Reporter:: Rajesh Balamohan

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 30/Mar/15 12:21

Updated:: 30/Jun/15 04:53

Resolved:: 03/Apr/15 00:46