[YARN-8418] App local logs could leaked if log aggregation fails to initialize for the app

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.8.0, 3.0.0-alpha1
• Fix Version/s: 3.2.0, 3.1.1
• Target Version/s: 3.1.1
• Hadoop Flags: Reviewed
• Assignee: Bibin Chundatt
• Reporter: Bibin Chundatt
• Created: 12/Jun/18 04:46
• Updated: 14/Apr/21 03:57
• Resolved: 31/Jul/18 19:30

Description

If log aggregation fails init createApp directory container logs could get leaked in NM directory

For log running application restart of NM after token renewal this case is possible/ Application submission with invalid delegation token

Issue Links

• is duplicated by: YARN-10734 Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
• is related to: YARN-5302 Yarn Application log Aggregation fails due to NM can not get correct HDFS delegation token II
• is related to: YARN-4984 LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak.