[YARN-5748] Backport YARN-5718 to branch-2

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Hadoop Flags: Incompatible change
• Assignee: Masatake Iwasaki
• Reporter: Junping Du
• Created: 18/Oct/16 01:47
• Updated: 25/Oct/19 20:25

Description

In YARN-5718, we have identify several unnecessary config to over-write HDFS client behavior in several components of YARN (FSRMStore, TimelineClient, NodeLabelStore, etc.) which cause job failure in some cases (NN HA, etc.) - that's definitely belongs to bug. In YARN-5718, we proposed to remove the config as it shouldn't be supposed to work, which get committed to trunk already as alpha stage has more flexibility for incompatible changes. In branch-2, we want to play a bit more safe and get more discussion.
Obviously, there are several options here:
1. Don't fix anything, let bug exist
2. Fix the bug, but keep the configuration, or mark it deprecated and add some explanation to say this configuration is not supposed to work any more.
3. Exactly like YARN-5718, fix the bug and remove the unnecessary configuration.
This ticket is filed for more discussion.

Issue Links

• relates to: YARN-5718 TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior