[TEZ-1942] Number of tasks show in Tez UI with auto-reduce parallelism is misleading

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.5.2
• Fix Version/s: 0.5.4
• Target Version/s: 0.6.0, 0.5.4
• Assignee: Prakash Ramachandran
• Reporter: Rajesh Balamohan
• Created: 14/Jan/15 03:52
• Updated: 30/Jun/15 04:53
• Resolved: 19/Jan/15 18:58

Description

Ran a simple hive query (with tez) and "--hiveconf hive.tez.auto.reducer.parallelism=true" . This internally turns on tez's auto reduce parallelism.

• Job started off with 1009 reduce tasks
• Tez reduces the number of reducers to 253
• Job completes successfully, but TEZ UI shows 1009 as the number of reducers (and 253 tasks as successful tasks). This can be a little misleading.

I will attach the screenshots soon.