Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9251

Allow a tag role:!overseer in replica placement rules

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.2
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      The reason to assign an overseer role to a node is to ensure that the node is exclusively used as overseer. replica placement should support tag called role

      So if a collection is created with rule=role:!overseer no replica should be created in nodes designated as overseer

        Activity

        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit df9fb16b46d6b9267364685f78236a8952d2d93a in lucene-solr's branch refs/heads/master from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=df9fb16 ]

        SOLR-9251: Support for a new tag 'role' in replica placement rules

        Show
        jira-bot ASF subversion and git services added a comment - Commit df9fb16b46d6b9267364685f78236a8952d2d93a in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=df9fb16 ] SOLR-9251 : Support for a new tag 'role' in replica placement rules
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5937247ac38aaaeb166b92fcc6f6a08681d73a1e in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5937247 ]

        SOLR-9251: Support for a new tag 'role' in replica placement rules

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5937247ac38aaaeb166b92fcc6f6a08681d73a1e in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5937247 ] SOLR-9251 : Support for a new tag 'role' in replica placement rules
        Hide
        mikemccand Michael McCandless added a comment -

        Bulk close resolved issues after 6.2.0 release.

        Show
        mikemccand Michael McCandless added a comment - Bulk close resolved issues after 6.2.0 release.
        Hide
        dsmiley David Smiley added a comment -

        Noble Paul does the rule system allow one to have this rule but specify further that a particular shard may go on this overseer node (or any designated node for that matter)? I hoped this would work:
        &rule=shard:RT,role:overseer
        &rule=shard:!RT,role:!overseer
        This is a collection with the implicit router and thus named shards; one of them named "RT". When I do this, Solr complains when attempting to create a replica that it could not identify nodes matching the rules. Note that this notion isn't specific to the overseer... I have also tried with host:specifcHostName and got the same result. I've tried various ways to try and achieve this but all in vain

        (apologies if you'd rather me ask on the user list)

        Show
        dsmiley David Smiley added a comment - Noble Paul does the rule system allow one to have this rule but specify further that a particular shard may go on this overseer node (or any designated node for that matter)? I hoped this would work: &rule=shard:RT,role:overseer &rule=shard:!RT,role:!overseer This is a collection with the implicit router and thus named shards; one of them named "RT". When I do this, Solr complains when attempting to create a replica that it could not identify nodes matching the rules. Note that this notion isn't specific to the overseer... I have also tried with host:specifcHostName and got the same result. I've tried various ways to try and achieve this but all in vain (apologies if you'd rather me ask on the user list)
        Hide
        noble.paul Noble Paul added a comment -

        It works only if you assign the rope of overseer to a particular node. By default, such a role does not exist. Refer to the ADDROLE command

        Show
        noble.paul Noble Paul added a comment - It works only if you assign the rope of overseer to a particular node. By default, such a role does not exist. Refer to the ADDROLE command
        Hide
        dsmiley David Smiley added a comment -

        Right; I should add I definitely did that and can confirm the overseer is running there.

        Show
        dsmiley David Smiley added a comment - Right; I should add I definitely did that and can confirm the overseer is running there.
        Hide
        noble.paul Noble Paul added a comment -

        the rule rule=shard:RT,role:overseer just does not look right. It means all replicas of shard RT must live in the node which is designated as overseer. why would you want all replicas of a given shard live in one node? it must be conflicting with maxShardsperNode ?

        Anyway, share your full create command

        Show
        noble.paul Noble Paul added a comment - the rule rule=shard:RT,role:overseer just does not look right. It means all replicas of shard RT must live in the node which is designated as overseer. why would you want all replicas of a given shard live in one node? it must be conflicting with maxShardsperNode ? Anyway, share your full create command
        Hide
        dsmiley David Smiley added a comment -

        Thanks for your help. I realize that most apps/use-cases call for additional replicas but mine doesn't – it's an identified and acceptable limitation for the confines of a slim operational budget. The system that can be reloaded if need be.

        To simplify this a bit, the example below uses a host tag instead of the role. I don't see an error but I do see a shard going where I don't want it to go. In particular, I want the "RT" shard on a specified host $rtHostName – this worked okay. But once I got to the 3rd S<num> shard, I saw it on the host rtHostName. I repeated this experiment after deleting the collection, and switching up which host is the designated RT host, and it observed this time it was the 4th numbered shard that was co-located with RT (the thing I'm trying to avoid), not the 3rd. Interesting. The cluster I am trying this on has 3 Solr nodes.

        curl -XPOST --fail "$SOLR_URL/admin/collections" -F action=CREATE -F name="$COLLECTION" \
          -F router.name=implicit -F shards=RT -F createNodeSet="${rtHostName}:8983_solr" -F maxShardsPerNode=4 \
          -F rule="shard:RT,host:$rtHostName" -F rule="shard:\!RT,host:\!$rtHostName"
        // note escaping of the exclaimations to make Bash happy
        
        curl -XPOST --fail "$SOLR_URL/admin/collections" -F action=CREATESHARD \
          -F collection="$COLLECTION" -F shard=s1
        
        //repeat above several times varying shard name: s1, s2, s3
        
        Show
        dsmiley David Smiley added a comment - Thanks for your help. I realize that most apps/use-cases call for additional replicas but mine doesn't – it's an identified and acceptable limitation for the confines of a slim operational budget. The system that can be reloaded if need be. To simplify this a bit, the example below uses a host tag instead of the role. I don't see an error but I do see a shard going where I don't want it to go. In particular, I want the "RT" shard on a specified host $rtHostName – this worked okay. But once I got to the 3rd S<num> shard, I saw it on the host rtHostName. I repeated this experiment after deleting the collection, and switching up which host is the designated RT host, and it observed this time it was the 4th numbered shard that was co-located with RT (the thing I'm trying to avoid), not the 3rd. Interesting. The cluster I am trying this on has 3 Solr nodes. curl -XPOST --fail "$SOLR_URL/admin/collections" -F action=CREATE -F name="$COLLECTION" \ -F router.name=implicit -F shards=RT -F createNodeSet="${rtHostName}:8983_solr" -F maxShardsPerNode=4 \ -F rule="shard:RT,host:$rtHostName" -F rule="shard:\!RT,host:\!$rtHostName" // note escaping of the exclaimations to make Bash happy curl -XPOST --fail "$SOLR_URL/admin/collections" -F action=CREATESHARD \ -F collection="$COLLECTION" -F shard=s1 //repeat above several times varying shard name: s1, s2, s3

          People

          • Assignee:
            noble.paul Noble Paul
            Reporter:
            noble.paul Noble Paul
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development