Solr
  1. Solr
  2. SOLR-7842

ZK connection loss or session expiry events should not fire config directory listeners

    Details

      Description

      The watcher on the config directory has the following in the process method:

      Stat stat = null;
            try {
              stat = zkClient.exists(zkDir, null, true);
            } catch (KeeperException e) {
              //ignore , it is not a big deal
            } catch (InterruptedException e) {
              Thread.currentThread().interrupt();
            }
      
            boolean resetWatcher = false;
            try {
              resetWatcher = fireEventListeners(zkDir);
            } finally {
              if (Event.EventType.None.equals(event.getType())) {
                log.info("A node got unwatched for {}", zkDir);
              } else {
                if (resetWatcher) setConfWatcher(zkDir, this, stat);
                else log.info("A node got unwatched for {}", zkDir);
              }
            }
      

      Even if the watcher is fired because of session expiry or connection loss, the fireEventListeners() method is executed and all subsequent listener invocations fail due to the loss of connection/session. All this is logged as well.

      466879 WARN  (Thread-78) [   ] o.a.s.c.ZkController listener throws error
      org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /configs/jepsen/params.json
              at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:158)
              at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:909)
              at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2585)
              at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2385)
      Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /configs/jepsen/params.json
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
              at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
              at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:302)
              at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:299)
              at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
              at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:299)
              at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)
              ... 3 more
      

      We should check the keeper state in addition to the event type and ignore such events.

      1. SOLR-7842.patch
        0.7 kB
        Shalin Shekhar Mangar

        Activity

        Hide
        Shalin Shekhar Mangar added a comment -

        Trivial patch which returns from watcher method if event state is disconnected or expired.

        Show
        Shalin Shekhar Mangar added a comment - Trivial patch which returns from watcher method if event state is disconnected or expired.
        Hide
        Noble Paul added a comment -

        +1

        Show
        Noble Paul added a comment - +1
        Hide
        ASF subversion and git services added a comment -

        Commit 1693218 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1693218 ]

        SOLR-7842: ZK connection loss or session expiry events should not fire config directory listeners

        Show
        ASF subversion and git services added a comment - Commit 1693218 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1693218 ] SOLR-7842 : ZK connection loss or session expiry events should not fire config directory listeners
        Hide
        ASF subversion and git services added a comment -

        Commit 1693219 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1693219 ]

        SOLR-7842: ZK connection loss or session expiry events should not fire config directory listeners

        Show
        ASF subversion and git services added a comment - Commit 1693219 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1693219 ] SOLR-7842 : ZK connection loss or session expiry events should not fire config directory listeners
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            Shalin Shekhar Mangar
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development