Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.6, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      To move a replica from a node to another node, there should be an API command. This should be better than having to do ADDREPLICA and DELETEREPLICA.
      The API will like this

      /admin/collections?action=MOVEREPLICA&collection=collection&shard=shard&replica=replica&node=nodeName&toNode=nodeName
      
      1. SOLR-10239.patch
        27 kB
        Cao Manh Dat
      2. SOLR-10239.patch
        27 kB
        Cao Manh Dat
      3. SOLR-10239.patch
        18 kB
        Cao Manh Dat

        Activity

        Hide
        noble.paul Noble Paul added a comment -

        This API can have 2 variants
        1) collection, replica and targetNode as parameters
        2) collection , shard, fromNode & targetNode

        Show
        noble.paul Noble Paul added a comment - This API can have 2 variants 1) collection , replica and targetNode as parameters 2) collection , shard , fromNode & targetNode
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        Should this API also be able to move all replicas from one node to another? Say the use-case is a node being replaced and we want to move all replicas that used to be on the old node to move to a new node? Or do we want to handle it in another issue (this use-case also has relations to Zk as Truth mode)

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - Should this API also be able to move all replicas from one node to another? Say the use-case is a node being replaced and we want to move all replicas that used to be on the old node to move to a new node? Or do we want to handle it in another issue (this use-case also has relations to Zk as Truth mode)
        Hide
        noble.paul Noble Paul added a comment -

        Shalin Shekhar Mangar We already have a command called REPLACENODE

        Show
        noble.paul Noble Paul added a comment - Shalin Shekhar Mangar We already have a command called REPLACENODE
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        Ah, you are right, I missed that.

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - Ah, you are right, I missed that.
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        In terms of design, I had couple of options in mind.

        1. ADDREPLICA on target node, followed by DELETEREPLICA on source node.
        2. Create a core on target node, fetch index from source node, then sync/recover with leader, finally DELETEREPLICA on source node.

        The former is simpler to build. The latter potentially exerts less pressure on the leader. Noble Paul, Shalin Shekhar Mangar, Cao Manh Dat, what do you think?

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - In terms of design, I had couple of options in mind. ADDREPLICA on target node, followed by DELETEREPLICA on source node. Create a core on target node, fetch index from source node, then sync/recover with leader, finally DELETEREPLICA on source node. The former is simpler to build. The latter potentially exerts less pressure on the leader. Noble Paul , Shalin Shekhar Mangar , Cao Manh Dat , what do you think?
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Noble Paul In case of #2, you mean we will randomly pickup a replica from fromNode and move to targetNode?

        Show
        caomanhdat Cao Manh Dat added a comment - Noble Paul In case of #2, you mean we will randomly pickup a replica from fromNode and move to targetNode ?
        Hide
        noble.paul Noble Paul added a comment -

        Randomly pick any replica of that given shard

        Show
        noble.paul Noble Paul added a comment - Randomly pick any replica of that given shard
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Noble Paul Although the logic is same but should we separate #2 into RANDOMLYMOVEREPLICA

        Show
        caomanhdat Cao Manh Dat added a comment - Noble Paul Although the logic is same but should we separate #2 into RANDOMLYMOVEREPLICA
        Hide
        noble.paul Noble Paul added a comment -

        No. No extra commands please

        Show
        noble.paul Noble Paul added a comment - No. No extra commands please
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Noble Paul In case of #2, you mean we will randomly pickup a replica from fromNode and move to targetNode?

        In a node, there will be only one replica for a particular shard. Is this not a correct understanding? If so, and given collection and shard is provided, why do we need to pick a replica on random?

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Noble Paul In case of #2, you mean we will randomly pickup a replica from fromNode and move to targetNode? In a node, there will be only one replica for a particular shard. Is this not a correct understanding? If so, and given collection and shard is provided, why do we need to pick a replica on random?
        Hide
        noble.paul Noble Paul added a comment -

        That's most likely the case. However, you are free to create more than one replica of a shard in a node

        Show
        noble.paul Noble Paul added a comment - That's most likely the case. However, you are free to create more than one replica of a shard in a node
        Hide
        caomanhdat Cao Manh Dat added a comment -

        First patch for this ticket.
        Noble Paul can you review this patch?

        Show
        caomanhdat Cao Manh Dat added a comment - First patch for this ticket. Noble Paul can you review this patch?
        Hide
        noble.paul Noble Paul added a comment -

        looks good

        This is not required. addReplica() would assign a core name anyway

         String newCoreName = Assign.buildCoreName(coll, slice.getName());
        
        Show
        noble.paul Noble Paul added a comment - looks good This is not required. addReplica() would assign a core name anyway String newCoreName = Assign.buildCoreName(coll, slice.getName());
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Noble Paul I just wanna to return the newCoreName in the result.

        Show
        caomanhdat Cao Manh Dat added a comment - Noble Paul I just wanna to return the newCoreName in the result.
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        This patch does not handle HDFS data directories. We should be able to delete the old replica and add a new one pointing it at the same index and tlog directories when using HDFS. This is essential to handle the use-case handled by autoAddReplica today.

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - This patch does not handle HDFS data directories. We should be able to delete the old replica and add a new one pointing it at the same index and tlog directories when using HDFS. This is essential to handle the use-case handled by autoAddReplica today.
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Shalin Shekhar Mangar Right now, we add a new replica first then delete the old replica. In your case, how can we add a new replica first ( the index dir is currently locked by the old replica )?

        Show
        caomanhdat Cao Manh Dat added a comment - Shalin Shekhar Mangar Right now, we add a new replica first then delete the old replica. In your case, how can we add a new replica first ( the index dir is currently locked by the old replica )?
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Updated patch for this ticket, added the optimization for hdfs case.
        Shalin Shekhar Mangar Can you review this patch?

        Show
        caomanhdat Cao Manh Dat added a comment - Updated patch for this ticket, added the optimization for hdfs case. Shalin Shekhar Mangar Can you review this patch?
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        I have only skimmed the patch without running it. Overall, it looks fine. Just that there is no need to ask the replica for its data dir using core admin API – in case of shared storage, the data and ulog directories are stored in cluster state itself – see SOLR-8913.

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - I have only skimmed the patch without running it. Overall, it looks fine. Just that there is no need to ask the replica for its data dir using core admin API – in case of shared storage, the data and ulog directories are stored in cluster state itself – see SOLR-8913 .
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Updated patch, based on Shalin Shekhar Mangar about get dataDir directly from clusterstate.

        Show
        caomanhdat Cao Manh Dat added a comment - Updated patch, based on Shalin Shekhar Mangar about get dataDir directly from clusterstate.
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        +1 LGTM

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - +1 LGTM
        Hide
        caomanhdat Cao Manh Dat added a comment -

        Thanks Shalin Shekhar Mangar, Noble Paul. I will commit it soon.

        Show
        caomanhdat Cao Manh Dat added a comment - Thanks Shalin Shekhar Mangar , Noble Paul . I will commit it soon.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9c2ef561e5fb17e9f74abbad62f3af0d81794b55 in lucene-solr's branch refs/heads/master from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9c2ef56 ]

        SOLR-10239: MOVEREPLICA API

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9c2ef561e5fb17e9f74abbad62f3af0d81794b55 in lucene-solr's branch refs/heads/master from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9c2ef56 ] SOLR-10239 : MOVEREPLICA API
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c30a7dc597f02f68c0f4b101a60e31b69d2d616e in lucene-solr's branch refs/heads/master from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c30a7dc ]

        SOLR-10239: change empty lambda to null

        Show
        jira-bot ASF subversion and git services added a comment - Commit c30a7dc597f02f68c0f4b101a60e31b69d2d616e in lucene-solr's branch refs/heads/master from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c30a7dc ] SOLR-10239 : change empty lambda to null
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c3bb6e20591d69e0f5039a56780a32d69c2543ec in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c3bb6e2 ]

        SOLR-10239: MOVEREPLICA API

        Show
        jira-bot ASF subversion and git services added a comment - Commit c3bb6e20591d69e0f5039a56780a32d69c2543ec in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c3bb6e2 ] SOLR-10239 : MOVEREPLICA API
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 78b84e65bd5262386917225cd94a90c078847ae6 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=78b84e6 ]

        SOLR-10239: change empty lambda to null

        Show
        jira-bot ASF subversion and git services added a comment - Commit 78b84e65bd5262386917225cd94a90c078847ae6 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=78b84e6 ] SOLR-10239 : change empty lambda to null
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit e0d0d98a5d465a5122231abc48989599f1dd6a15 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e0d0d98 ]

        SOLR-10239: Update CHANGES.txt

        Show
        jira-bot ASF subversion and git services added a comment - Commit e0d0d98a5d465a5122231abc48989599f1dd6a15 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e0d0d98 ] SOLR-10239 : Update CHANGES.txt
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ad1326698d600e89c6f745f3817acc4b6a3f84e7 in lucene-solr's branch refs/heads/master from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ad13266 ]

        SOLR-10239: Update CHANGES.txt

        Show
        jira-bot ASF subversion and git services added a comment - Commit ad1326698d600e89c6f745f3817acc4b6a3f84e7 in lucene-solr's branch refs/heads/master from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ad13266 ] SOLR-10239 : Update CHANGES.txt
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit a786f2eb4e6157334c516a2fba98f93cef800dc2 in lucene-solr's branch refs/heads/master from Cassandra Targett
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a786f2e ]

        Ref Guide: Placeholders for docs for SOLR-10239 & SOLR-10447

        Show
        jira-bot ASF subversion and git services added a comment - Commit a786f2eb4e6157334c516a2fba98f93cef800dc2 in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a786f2e ] Ref Guide: Placeholders for docs for SOLR-10239 & SOLR-10447
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5d0aa603d724f4e2d9aa2a62883c240b0631f2dd in lucene-solr's branch refs/heads/master from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5d0aa60 ]

        SOLR-10239: Adding documentation for MoveReplica

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5d0aa603d724f4e2d9aa2a62883c240b0631f2dd in lucene-solr's branch refs/heads/master from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5d0aa60 ] SOLR-10239 : Adding documentation for MoveReplica
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 39c43ecb13f53d27db26b36f1139e06347852295 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=39c43ec ]

        SOLR-10239: Adding documentation for MoveReplica

        Show
        jira-bot ASF subversion and git services added a comment - Commit 39c43ecb13f53d27db26b36f1139e06347852295 in lucene-solr's branch refs/heads/branch_6x from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=39c43ec ] SOLR-10239 : Adding documentation for MoveReplica
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b09cc9164cf47892c99809db5f8e10af919e232a in lucene-solr's branch refs/heads/branch_6_6 from Cao Manh Dat
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b09cc91 ]

        SOLR-10239: Adding documentation for MoveReplica

        Show
        jira-bot ASF subversion and git services added a comment - Commit b09cc9164cf47892c99809db5f8e10af919e232a in lucene-solr's branch refs/heads/branch_6_6 from Cao Manh Dat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b09cc91 ] SOLR-10239 : Adding documentation for MoveReplica

          People

          • Assignee:
            caomanhdat Cao Manh Dat
            Reporter:
            ichattopadhyaya Ishan Chattopadhyaya
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development