[SPARK-10916] YARN executors are launched with the default perm gen size

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.1, 1.6.0
• Fix Version/s: 1.6.0
• Component/s: YARN
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 02/Oct/15 21:05
• Updated: 06/Oct/15 17:17
• Resolved: 06/Oct/15 17:17

Description

Unlike other backends, the YARN one does not explicitly set the perm gen size for the executor process. That means that, unless the user has explicitly changed it by adding extra java options, executors on YARN are running with 64m of perm gen (I believe) instead of 256m like the other backends.

Issue Links

• links to: [Github] Pull Request #8970 (vanzin)