[TEZ-376] Reduce Processor can parallelize I/O/Processor initialization

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Labels: TEZ-0.2.0
• Assignee: Siddharth Seth
• Reporter: Siddharth Seth
• Created: 19/Aug/13 18:43
• Updated: 01/Dec/13 20:21
• Resolved: 20/Aug/13 18:44

Description

Reported by gopalv. Reduce initialization can take a fair amount of time after shuffle is complete - depending on how expensive the Reduce initialization is.

Quote from Gopal's investigation
1.6 seconds is spent in ReflectionUtils.newInstance() -> ReflectionUtils.setConf() of which 1.2 seconds is Utilities.getMapRedWork().