Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8931

SolrCloud RebalanceShards API

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • None
    • 6.0
    • SolrCloud

    Description

      It would be great to have RebalanceShards action in SolrCloud, such like described in this post by Suruchi Shah : "http://engineering.bloomreach.com/solrcloud-rebalance-api/"

      By the way, in order to rebalance shards from a collection with replicationFactor > 1, one idea could be to split some shards using the rule-based replica placement.
      Since https://issues.apache.org/jira/browse/SOLR-8728 jira issue, splitShard is using rule-based replica placement (for the "replication" replicas).

      As part of a proof of concept, the attached patch introduce a new action to the collections API, named "REBALANCESHARDS", to rebalance some or all shards among solrCloud nodes using splitShard.
      After each splitShard, a deleteshard of the inactive parent shard is done.
      One mandatory parameter:

      • collection: the name of the collection
        Two parameters:
      • deltaMaxFromAverage (default: 20): use to select n shards (<= half of all shards) to be split, whose number of docs are greater than deltaMaxFromAverage percent of the average number of docs per shard
      • force (default: false): if true, in case of none shards selected with the deltaMaxFromAverage given, all shards of the collection will be selected to be split

      Use example:
      curl 'http://ip:port/solr/admin/collections?action=REBALANCESHARDS&collection=collection1&async=2&deltaMaxFromAverage=30'

      Drawbacks: replicationFactor must be more than one, select shards based on the average number of docs per shard is not suitable for all cases

      Attachments

        1. SOLR-8931.patch
          20 kB
          olivier soyez

        Issue Links

          Activity

            People

              Unassigned Unassigned
              soyouz olivier soyez
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: