[SPARK-37215] Support Application Timeouts on YARN

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: YARN
• Assignee: Unassigned
• Reporter: Kent Yao 2
• Created: 05/Nov/21 09:23
• Updated: 05/Nov/21 09:41

Description

Since #YARN-3813/2.9.0/3.0.0, YARN supports ApplicationTimeouts. It helps enforce lifetime application SLAs. Currently, lifetime indicates the overall time spent by an application in YARN. It is calculated from its submit time to finish time, including running time and the waiting time for resource allocation.

YARN allows admins to set lifetime of an application at leaf-queue. It also allows users to set it programmatically. During application submission, user can set it in  ApplicationSubmissionContext#setApplicationTimeouts(Map<ApplicationTimeoutType, Long> applicationTimeouts).

 

So far, YARN supports for one timeout type - LIFETIME 

Issue Links

• links to: [Github] Pull Request #34491 (yaooqinn)
• links to: [Github] Pull Request #34491 (yaooqinn)