Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.7, Trunk
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

      It works as a new collection admin command

      command=addrole&role=overseer&node=192.168.1.5:8983_solr

      This results in the creation of a entry in the /roles.json in ZK which would look like the following

      {
      "overseer" : ["192.168.1.5:8983_solr"]
      }
      

      If a node is designated for overseer it gets preference over others when overseer election takes place. If no designated servers are available another random node would become the Overseer.

      Later on, if one of the designated nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

      1. SOLR-5476.patch
        17 kB
        Noble Paul
      2. SOLR-5476.patch
        23 kB
        Noble Paul
      3. SOLR-5476.patch
        24 kB
        Noble Paul
      4. SOLR-5476.patch
        25 kB
        Noble Paul
      5. SOLR-5476.patch
        25 kB
        Noble Paul
      6. SOLR-5476.patch
        5 kB
        Noble Paul
      7. SOLR-5476.patch
        5 kB
        Noble Paul

        Issue Links

          Activity

          Noble Paul made changes -
          Link This issue is related to SOLR-6095 [ SOLR-6095 ]
          Noble Paul made changes -
          Link This issue relates to SOLR-5893 [ SOLR-5893 ]
          David Smiley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Noble Paul made changes -
          Link This issue relates to SOLR-5859 [ SOLR-5859 ]
          Noble Paul made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12627977 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12627815 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12627814 ]
          Noble Paul made changes -
          Comment [ This is to avoid the race condition

          In this patch
          * The new overseer is asked to wait for the STATE_UPDATE_DELAY before processing the queue items
          * STATE_UPDATE_DELAY is the maximum interval between the Overseer checks for amILeader().
          ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12627814 ]
          Mark Miller made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Noble Paul made changes -
          Fix Version/s 5.0 [ 12321664 ]
          Fix Version/s 4.7 [ 12325573 ]
          Noble Paul made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12623354 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12623336 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12623336 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12621791 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12621609 ]
          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12621324 ]
          Noble Paul made changes -
          Description In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=192.168.1.5:8983_solr&node=192.168.1.6:8983_solr

          This results in the creation of a entry in the /roles.json in ZK which would look like the following

          {code:javascript}
          {
          "overseer" : ["192.168.1.5:8983_solr", "192.168.1.6:8983_solr"]
          }
          {code}
          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node would become the Overseer.

          Later on, if one of the whitelisted nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

          In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=addrole&role=overseer&node=192.168.1.5:8983_solr

          This results in the creation of a entry in the /roles.json in ZK which would look like the following

          {code:javascript}
          {
          "overseer" : ["192.168.1.5:8983_solr"]
          }
          {code}
          If a node is designated for overseer it gets preference over others when overseer election takes place. If no designated servers are available another random node would become the Overseer.

          Later on, if one of the designated nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

          Noble Paul made changes -
          Attachment SOLR-5476.patch [ 12620829 ]
          Noble Paul made changes -
          Description In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=192.168.1.5:8983_solr&node=192.168.1.6:8983_solr

          This results in the creation of a entry in the /roles.json in ZK which would look like the following


          {
          "overseer" : {
                            "whitelist":["192.168.1.5:8983_solr", "192.168.1.6:8983_solr"]
                            }

          }

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node would become the Overseer.

          Later on, if one of the whitelisted nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

          In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=192.168.1.5:8983_solr&node=192.168.1.6:8983_solr

          This results in the creation of a entry in the /roles.json in ZK which would look like the following

          {code:javascript}
          {
          "overseer" : ["192.168.1.5:8983_solr", "192.168.1.6:8983_solr"]
          }
          {code}
          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node would become the Overseer.

          Later on, if one of the whitelisted nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

          Noble Paul made changes -
          Assignee Noble Paul [ noble.paul ]
          Noble Paul made changes -
          Description In a very large cluster the OverSeer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=node1_name&node=node2_name

          This results in the creation of a entry in the /roles.json in ZK which would look like the following


          {
          "overseer" : {

                            }

          }

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node will be picked up


          In a very large cluster the Overseer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=192.168.1.5:8983_solr&node=192.168.1.6:8983_solr

          This results in the creation of a entry in the /roles.json in ZK which would look like the following


          {
          "overseer" : {
                            "whitelist":["192.168.1.5:8983_solr", "192.168.1.6:8983_solr"]
                            }

          }

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node would become the Overseer.

          Later on, if one of the whitelisted nodes are brought up ,it would take over the Overseer role from the current Overseer to become the Overseer of the system

          Noble Paul made changes -
          Description In a very large cluster the OverSeer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=node1_name&node=node2_name

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node will be picked up

          In a very large cluster the OverSeer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=node1_name&node=node2_name

          This results in the creation of a entry in the /roles.json in ZK which would look like the following


          {
          "overseer" : {

                            }

          }

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node will be picked up


          Noble Paul made changes -
          Summary Roles per node Overseer Role for nodes
          Noble Paul made changes -
          Description In a very large cluster the OverSeer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&blacklist=leader&blacklist=replica&node=node_name

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node will be picked up

          if the node is blacklisted for leade/replica , it won't be assigned any new shards
          In a very large cluster the OverSeer is likely to be overloaded.If the same node is a serving a few other shards it can lead to OverSeer getting slowed down due to GC pauses , or simply too much of work . If the cluster is really large , it is possible to dedicate high end h/w for OverSeers

          It works as a new collection admin command

          command=assignRole&whitelist=overseer&node=node1_name&node=node2_name

          If a node is whitelisted for overseer it gets preference over others when overseer election takes place. If no whitelisted servers are available another random node will be picked up

          Noble Paul made changes -
          Field Original Value New Value
          Component/s SolrCloud [ 12313841 ]
          Noble Paul created issue -

            People

            • Assignee:
              Noble Paul
              Reporter:
              Noble Paul
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development