[TEZ-1618] LocalTaskSchedulerService.getTotalResources() and getAvailableResources() can get negative if JVM memory is larger than 2GB

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.1
• Assignee: Chen He
• Reporter: Chen He
• Created: 23/Sep/14 20:12
• Updated: 02/Oct/14 21:40
• Resolved: 25/Sep/14 19:15

Description

Tez job can get failed because of negative headroom value;

org.apache.tez.dag.app.dag.impl.VertexImpl - Creating 2 for vertex:
vertex_1411054289802_0001_1_01 [intermediate_reducer]
56389 InputInitializer [initialmap] #0 INFO
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator - Input MRInput asking
for -2 tasks. strong Headroom: -1365 strong Task Resource: 1024 waves: 1.7
56391 InputInitializer [initialmap] #0 INFO
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator - Grouping input splits
56391 InputInitializer [initialmap] #0 INFO
org.apache.tez.mapreduce.hadoop.MRInputHelpers - Generating mapreduce api input
splits
56400 [AsyncDispatcher event handler] INFO
org.apache.tez.dag.app.dag.impl.VertexImpl - Directly initializing vertex:
vertex_1411054289802_0001_1_01 [intermediate_reducer]