[SPARK-3476] Yarn ClientBase.validateArgs memory checks wrong

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Component/s: YARN
• Target Version/s: 1.2.0
• Assignee: Andrew Or
• Reporter: Thomas Graves
• Created: 10/Sep/14 20:13
• Updated: 26/Sep/14 18:51
• Resolved: 26/Sep/14 18:51

Description

The yarn ClientBase.validateArgs memory checks are no longer valid. It used to be that the overhead was taken out of what the user specified, now we add it on top of what the user specifies. We can probably just remove these.

(args.amMemory <= memoryOverhead) -> ("Error: AM memory size must be" +
"greater than: " + memoryOverhead),
(args.executorMemory <= memoryOverhead) -> ("Error: Executor memory size" +
"must be greater than: " + memoryOverhead.toString)

Issue Links

• links to: [Github] Pull Request #2528 (andrewor14)