Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.6, 6.0
    • SolrCloud
    • None

    Description

      Provide a way to split a shard using a route key such that all documents of the specified route key end up in a single dedicated sub-shard.

      Example:
      Assume that collection1, shard1 has hash range [0, 20]. Also that route key 'A!' has hash range [12,15]. Then invoking:

      /admin/collections?action=SPLIT&collection=collection1&split.key=A!
      

      should produce three sub-shards with hash range [0,11], [12,15] and [16,20].

      Specifying the source shard is not required here because the route key is enough to figure it out. Route keys spanning more than one shards will not be supported.

      Note that the sub-shard with the hash range of the route key may also contain documents for other route keys whose hash ranges overlap.

      Attachments

        1. SOLR-5338-remove-exception.patch
          2 kB
          Shalin Shekhar Mangar
        2. SOLR-5338.patch
          20 kB
          Shalin Shekhar Mangar
        3. SOLR-5338.patch
          20 kB
          Shalin Shekhar Mangar
        4. SOLR-5338.patch
          18 kB
          Shalin Shekhar Mangar

        Issue Links

          Activity

            Changes:

            • Introduces two new methods in CompositeIdRouter
              public List<Range> partitionRangeByKey(String key, Range range)
              

              and

              public Range routeKeyHashRange(String routeKey)
              
            • The collection split action accepts a new parameter 'split.key'
            • The parent slice is found and its range is partitioned according to split.key
            • We re-use the logic introduced in SOLR-5300 to do the actual splitting.
            shalin Shalin Shekhar Mangar added a comment - Changes: Introduces two new methods in CompositeIdRouter public List<Range> partitionRangeByKey( String key, Range range) and public Range routeKeyHashRange( String routeKey) The collection split action accepts a new parameter 'split.key' The parent slice is found and its range is partitioned according to split.key We re-use the logic introduced in SOLR-5300 to do the actual splitting.

            yseeley@gmail.com - Would you mind reviewing the new CompositeIdRouter methods?

            shalin Shalin Shekhar Mangar added a comment - yseeley@gmail.com - Would you mind reviewing the new CompositeIdRouter methods?

            Fixed bug a in the test related to counting documents with a particular route key.

            shalin Shalin Shekhar Mangar added a comment - Fixed bug a in the test related to counting documents with a particular route key.
            1. Detect and abort if split.key's hash range is equal to parent shard's hash range
            2. Removed useless test added in SolrIndexSplitter. The one in ShardSplitTest is better.
            shalin Shalin Shekhar Mangar added a comment - Detect and abort if split.key's hash range is equal to parent shard's hash range Removed useless test added in SolrIndexSplitter. The one in ShardSplitTest is better.

            Commit 1531845 from shalin@apache.org in branch 'dev/trunk'
            [ https://svn.apache.org/r1531845 ]

            SOLR-5338: Split shards by a route key using split.key parameter

            jira-bot ASF subversion and git services added a comment - Commit 1531845 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1531845 ] SOLR-5338 : Split shards by a route key using split.key parameter

            Commit 1532862 from shalin@apache.org in branch 'dev/branches/branch_4x'
            [ https://svn.apache.org/r1532862 ]

            SOLR-5338: Split shards by a route key using split.key parameter

            jira-bot ASF subversion and git services added a comment - Commit 1532862 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1532862 ] SOLR-5338 : Split shards by a route key using split.key parameter
            yseeley@gmail.com Yonik Seeley added a comment -

            Would you mind reviewing the new CompositeIdRouter methods?

            Yeah, these make sense. You could optionally not throw this error:
            + throw new IllegalArgumentException("Route key must be a composite id");

            and just return a range that covers the single hash code of the ID.

            yseeley@gmail.com Yonik Seeley added a comment - Would you mind reviewing the new CompositeIdRouter methods? Yeah, these make sense. You could optionally not throw this error: + throw new IllegalArgumentException("Route key must be a composite id"); and just return a range that covers the single hash code of the ID.

            You could optionally not throw this error:
            + throw new IllegalArgumentException("Route key must be a composite id");
            and just return a range that covers the single hash code of the ID.

            Patch to remove the exception and return a Range with a single hash. I moved this exception handling to OverseerCollectionProcessor.

            Thanks Yonik!

            shalin Shalin Shekhar Mangar added a comment - You could optionally not throw this error: + throw new IllegalArgumentException("Route key must be a composite id"); and just return a range that covers the single hash code of the ID. Patch to remove the exception and return a Range with a single hash. I moved this exception handling to OverseerCollectionProcessor. Thanks Yonik!

            Commit 1532867 from shalin@apache.org in branch 'dev/trunk'
            [ https://svn.apache.org/r1532867 ]

            SOLR-5338: CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

            jira-bot ASF subversion and git services added a comment - Commit 1532867 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1532867 ] SOLR-5338 : CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

            Commit 1532868 from shalin@apache.org in branch 'dev/branches/branch_4x'
            [ https://svn.apache.org/r1532868 ]

            SOLR-5338: CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

            jira-bot ASF subversion and git services added a comment - Commit 1532868 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1532868 ] SOLR-5338 : CompositeIdRouter.keyHashRange should not throw exception if route key is not composite

            People

              shalin Shalin Shekhar Mangar
              shalin Shalin Shekhar Mangar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: