Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14927

Remove Overseer

    XMLWordPrintableJSON

    Details

      Description

      This Jira is intended to capture sub jiras on the path to remove the Overseer component from SolrCloud and move to all nodes being able to do the work currently done by Overseer.

      See detailed description in this doc.

      Copying (edited) from the above doc:

      The motivation for removing Overseer include:

      • Mono threaded state change is slow and doesn’t scale,
      • Communication between cluster nodes and the Overseer use Zookeeper as a queueing mechanism, this is not a good idea,
      • Nodes talking to Overseer (then Overseer talking to itself) via Zookeeper is inefficient and adds latency,
      • Collection API scalability is poor, because not only a single node processes commands for all Collections, but it also depends on the mono threaded state change queue consumption,
      • The code supporting Overseer in SolrCloud is complex (election, queue management, recovery etc).

      The general idea is that there’s already a central point in the SolrCloud cluster and it’s Zookeeper. It might not be necessary to have a second central point (Overseer) because nodes can interact directly with Zookeeper and synchronize more efficiently by optimistic locking using “conditional updates” (a.k.a compare and swap or CAS).

       

        Attachments

          Activity

            People

            • Assignee:
              ilan Ilan Ginzburg
              Reporter:
              ilan Ilan Ginzburg
            • Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 6h 10m
                6h 10m