[YARN-3469] ZKRMStateStore: Avoid setting watches that are not required

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 2.7.1, 3.0.0-alpha1
• Hadoop Flags: Reviewed
• Assignee: Jun Gong
• Reporter: Jun Gong
• Created: 09/Apr/15 13:32
• Updated: 15/May/17 05:21
• Resolved: 10/Apr/15 18:25

Description

In ZKRMStateStore, most operations(e.g. getDataWithRetries, getDataWithRetries, getDataWithRetries) set watches on znode. Large watches will cause problem such as ZOOKEEPER-706: large numbers of watches can cause session re-establishment to fail.

Although there is a workaround that setting jute.maxbuffer to a larger value, we need to adjust this value once there are more app and attempts stored in ZK. And those watches are useless now. It might be better that do not set watches.

Issue Links

• duplicates: YARN-3795 ZKRMStateStore crashes due to IOException: Broken pipe
• is blocked by: YARN-3795 ZKRMStateStore crashes due to IOException: Broken pipe
• is related to: YARN-3604 removeApplication in ZKRMStateStore should also disable watch.