[FLINK-12472] Support setting attemptFailuresValidityInterval of jobs on Yarn

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: Deployment / YARN
• Labels: pull-request-available
• Release Note: Yarn's attempt failure validity interval no longer defaults to the ask timeout `akka.ask.timeout`. Instead it can now be configured independently via `yarn.application-attempt-failures-validity-interval` whose default value is `10000` milliseconds.
• Assignee: jiasheng55
• Reporter: jiasheng55
• Due: 10/May/19
• Created: 10/May/19 02:48
• Updated: 18/Feb/22 06:23
• Resolved: 14/May/19 09:47

Description

According to the documentation of Yarn, a yarn application can set a attemptFailuresValidityInterval  to reset application attempts.

 

"attemptFailuresValidityInterval. The default value is -1. when attemptFailuresValidityInterval in milliseconds is set to > 0, the failure number will no take failures which happen out of the validityInterval into failure count. If failure count reaches to maxAppAttempts, the application will be failed."

 

We can make use of this feature to make Flink jobs on Yarn to be more long-running.

Issue Links

• is duplicated by: FLINK-10898 Add yarn.application-attempts-failures-validity-interval in YarnConfigOptions
• links to: GitHub Pull Request #8400