Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.6, 5.0
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      Provide a way to split a shard using a route key such that all documents of the specified route key end up in a single dedicated sub-shard.

      Example:
      Assume that collection1, shard1 has hash range [0, 20]. Also that route key 'A!' has hash range [12,15]. Then invoking:

      /admin/collections?action=SPLIT&collection=collection1&split.key=A!
      

      should produce three sub-shards with hash range [0,11], [12,15] and [16,20].

      Specifying the source shard is not required here because the route key is enough to figure it out. Route keys spanning more than one shards will not be supported.

      Note that the sub-shard with the hash range of the route key may also contain documents for other route keys whose hash ranges overlap.

      1. SOLR-5338.patch
        18 kB
        Shalin Shekhar Mangar
      2. SOLR-5338.patch
        20 kB
        Shalin Shekhar Mangar
      3. SOLR-5338.patch
        20 kB
        Shalin Shekhar Mangar
      4. SOLR-5338-remove-exception.patch
        2 kB
        Shalin Shekhar Mangar

        Issue Links

          Activity

          Hide
          ASF subversion and git services added a comment -

          Commit 1532868 from shalin@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1532868 ]

          SOLR-5338: CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

          Show
          ASF subversion and git services added a comment - Commit 1532868 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1532868 ] SOLR-5338 : CompositeIdRouter.keyHashRange should not throw exception if route key is not composite
          Hide
          ASF subversion and git services added a comment -

          Commit 1532867 from shalin@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1532867 ]

          SOLR-5338: CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

          Show
          ASF subversion and git services added a comment - Commit 1532867 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1532867 ] SOLR-5338 : CompositeIdRouter.keyHashRange should not throw exception if route key is not composite
          Hide
          Shalin Shekhar Mangar added a comment -

          You could optionally not throw this error:
          + throw new IllegalArgumentException("Route key must be a composite id");
          and just return a range that covers the single hash code of the ID.

          Patch to remove the exception and return a Range with a single hash. I moved this exception handling to OverseerCollectionProcessor.

          Thanks Yonik!

          Show
          Shalin Shekhar Mangar added a comment - You could optionally not throw this error: + throw new IllegalArgumentException("Route key must be a composite id"); and just return a range that covers the single hash code of the ID. Patch to remove the exception and return a Range with a single hash. I moved this exception handling to OverseerCollectionProcessor. Thanks Yonik!
          Hide
          Yonik Seeley added a comment -

          Would you mind reviewing the new CompositeIdRouter methods?

          Yeah, these make sense. You could optionally not throw this error:
          + throw new IllegalArgumentException("Route key must be a composite id");

          and just return a range that covers the single hash code of the ID.

          Show
          Yonik Seeley added a comment - Would you mind reviewing the new CompositeIdRouter methods? Yeah, these make sense. You could optionally not throw this error: + throw new IllegalArgumentException("Route key must be a composite id"); and just return a range that covers the single hash code of the ID.
          Hide
          ASF subversion and git services added a comment -

          Commit 1532862 from shalin@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1532862 ]

          SOLR-5338: Split shards by a route key using split.key parameter

          Show
          ASF subversion and git services added a comment - Commit 1532862 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1532862 ] SOLR-5338 : Split shards by a route key using split.key parameter
          Hide
          ASF subversion and git services added a comment -

          Commit 1531845 from shalin@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1531845 ]

          SOLR-5338: Split shards by a route key using split.key parameter

          Show
          ASF subversion and git services added a comment - Commit 1531845 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1531845 ] SOLR-5338 : Split shards by a route key using split.key parameter
          Hide
          Shalin Shekhar Mangar added a comment -
          1. Detect and abort if split.key's hash range is equal to parent shard's hash range
          2. Removed useless test added in SolrIndexSplitter. The one in ShardSplitTest is better.
          Show
          Shalin Shekhar Mangar added a comment - Detect and abort if split.key's hash range is equal to parent shard's hash range Removed useless test added in SolrIndexSplitter. The one in ShardSplitTest is better.
          Hide
          Shalin Shekhar Mangar added a comment -

          Fixed bug a in the test related to counting documents with a particular route key.

          Show
          Shalin Shekhar Mangar added a comment - Fixed bug a in the test related to counting documents with a particular route key.
          Hide
          Shalin Shekhar Mangar added a comment -

          Yonik Seeley - Would you mind reviewing the new CompositeIdRouter methods?

          Show
          Shalin Shekhar Mangar added a comment - Yonik Seeley - Would you mind reviewing the new CompositeIdRouter methods?
          Hide
          Shalin Shekhar Mangar added a comment -

          Changes:

          • Introduces two new methods in CompositeIdRouter
            public List<Range> partitionRangeByKey(String key, Range range)
            

            and

            public Range routeKeyHashRange(String routeKey)
            
          • The collection split action accepts a new parameter 'split.key'
          • The parent slice is found and its range is partitioned according to split.key
          • We re-use the logic introduced in SOLR-5300 to do the actual splitting.
          Show
          Shalin Shekhar Mangar added a comment - Changes: Introduces two new methods in CompositeIdRouter public List<Range> partitionRangeByKey( String key, Range range) and public Range routeKeyHashRange( String routeKey) The collection split action accepts a new parameter 'split.key' The parent slice is found and its range is partitioned according to split.key We re-use the logic introduced in SOLR-5300 to do the actual splitting.

            People

            • Assignee:
              Shalin Shekhar Mangar
              Reporter:
              Shalin Shekhar Mangar
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development