[SPARK-20884] Spark' masters will be both standby due to the bug of curator

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 2.1.1
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: liuzhaokun
• Created: 25/May/17 11:59
• Updated: 26/Nov/19 14:42
• Resolved: 06/Jun/17 09:30

Description

I built a cluster with two masters and three workers.When there is a switch of master's state,two masters are both standby in a long period.There are some ERROR in master's logfile " ERROR CuratorFrameworkImpl: Background exception was not retry-able or retry gave up
java.lang.IllegalArgumentException: Path must start with / character
at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:53)
at org.apache.curator.utils.ZKPaths.getNodeFromPath(ZKPaths.java:56)
at org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:421)
at org.apache.curator.framework.recipes.leader.LeaderLatch.access$500(LeaderLatch.java:60)
at org.apache.curator.framework.recipes.leader.LeaderLatch$6.processResult(LeaderLatch.java:478)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:686)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:485)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:587)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)"
it can be related to https://issues.apache.org/jira/browse/CURATOR-168.
So I think the version of curator in spark should be 2.8.

Issue Links

• relates to: SPARK-13933 hadoop-2.7 profile's curator version should be 2.7.1
• relates to: CURATOR-168 Background protected mode incorrectly sets Event name to just ZNode name. Should be full path.
• links to: [Github] Pull Request #18108 (liu-zhaokun)