Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14927 Remove Overseer
  3. SOLR-15146

Distribute Collection API command execution

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: main (9.0)
    • Fix Version/s: None
    • Component/s: SolrCloud

      Description

      Building on the distributed cluster state update changes (SOLR-14928), this ticket will distribute the Collection API so that commands can execute on any node (i.e. the node handling the request through CollectionsHandler) without having to go through a Zookeeper queue and the Overseer.
      This is the second step (first was SOLR-14928) after which the Overseer could be removed (but the code keeps existing execution options so completion by no means Overseer is gone, but it could be removed in a future release).

      There is a dependency on the distributed cluster state changes because the Overseer locking protecting same collection (or same shard) Collection API commands from executing concurrently will be replaced by optimistic locking of the collection state.json znodes (or other znodes that will eventually replace/augment state.json).

      The goal of this ticket is threefold:

      • Simplify the code (running synchronously and not going through the Zookeeper queues and the Overseer dequeue logic is much simpler),
      • Lead to improved performance for most/all use cases (although this is a secondary goal, as long as performance is not degraded) and
      • Allow a future change (in another future Jira) to the way cluster state is cached on the nodes of the cluster (keep less information, be less dependent on Zookeeper watches, do not care about collections not present on the node). This future work will aim to significantly increase the scale (amount of collections) supported by SolrCloud.

        Attachments

          Activity

            People

            • Assignee:
              ilan Ilan Ginzburg
              Reporter:
              ilan Ilan Ginzburg
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m