[YARN-3760] FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close()

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
• Component/s: nodemanager
• Target Version/s: 2.8.1, 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Haibo Chen
• Reporter: Daryn Sharp
• Created: 02/Jun/15 15:27
• Updated: 25/Oct/19 20:24
• Resolved: 12/Apr/17 21:21

Description

The aggregated log file does not appear to be properly closed when writes fail. This leaves a lease renewer active in the NM that spams the NN with lease renewals. If the token is marked not to be cancelled, the renewals appear to continue until the token expires. If the token is cancelled, the periodic renew spam turns into a flood of failed connections until the lease renewer gives up.

Issue Links

• relates to: HADOOP-12054 RPC client should not retry for InvalidToken exceptions
• relates to: YARN-6288 Exceptions during aggregated log writes are mishandled
• relates to: HDFS-11590 Nodemanagers have DDoS our namenode due to HDFS_DELEGATION_TOKEN expired or not in the cache