Details

    • Type: Umbrella Umbrella
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Umbrella issue under which we hang all regions related to balancer

        Issue Links

          Activity

          stack created issue -
          stack made changes -
          Field Original Value New Value
          Link This issue incorporates HBASE-3373 [ HBASE-3373 ]
          stack made changes -
          Link This issue incorporates HBASE-3679 [ HBASE-3679 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-4191 [ HBASE-4191 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-4464 [ HBASE-4464 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-3803 [ HBASE-3803 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-3815 [ HBASE-3815 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-3799 [ HBASE-3799 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-3609 [ HBASE-3609 ]
          gaojinchao made changes -
          Link This issue relates to HBASE-3586 [ HBASE-3586 ]
          Hide
          gaojinchao added a comment -

          I found the balance ago in branch92 is invalid for our scenario.
          So I use this issue to hang all issues related to balance. If someone want to see it,
          it will be easy.

          Show
          gaojinchao added a comment - I found the balance ago in branch92 is invalid for our scenario. So I use this issue to hang all issues related to balance. If someone want to see it, it will be easy.
          Hide
          Ted Yu added a comment -

          @Jinchao:
          Can you describe your scenario ?
          Then we will see which task can best accommodate your requirement.

          Show
          Ted Yu added a comment - @Jinchao: Can you describe your scenario ? Then we will see which task can best accommodate your requirement.
          Hide
          gaojinchao added a comment -

          @zhihong

          In my scenario:

          1. One region server has more than 1,000 regions.(hdf hard disk capacity(12T) / 3 (replication) /
          region size = 2G).

          2. One moment, Dozens of Regions per region server are working for put operation.

          I went through the balance code , I found current balnace algorithm is invalid for our scenarios.

          below scenarios:
          1)When adds one new machine to our cluster, Maybe all of hot regions(is working) will move to this one.
          2)When one RS restarts, Maybe all of hot regions(is working) will move to this machine.

          Show
          gaojinchao added a comment - @zhihong In my scenario: 1. One region server has more than 1,000 regions.(hdf hard disk capacity(12T) / 3 (replication) / region size = 2G). 2. One moment, Dozens of Regions per region server are working for put operation. I went through the balance code , I found current balnace algorithm is invalid for our scenarios. below scenarios: 1)When adds one new machine to our cluster, Maybe all of hot regions(is working) will move to this one. 2)When one RS restarts, Maybe all of hot regions(is working) will move to this machine.
          Hide
          Ted Yu added a comment -

          @Jinchao:
          Are all regions created at time of table creation ?
          If not, the balancer code in 0.92 is able to move both hot and cold regions onto the new server.

          Show
          Ted Yu added a comment - @Jinchao: Are all regions created at time of table creation ? If not, the balancer code in 0.92 is able to move both hot and cold regions onto the new server.
          Hide
          gaojinchao added a comment -

          for example:
          we have 10 nodes cluster. 1000 regions per node and 50 regions are hot.
          per the current algorithm.

          If we add a new machine to this cluster. all the 50 hot regions will be moved to the new machine?

          Show
          gaojinchao added a comment - for example: we have 10 nodes cluster. 1000 regions per node and 50 regions are hot. per the current algorithm. If we add a new machine to this cluster. all the 50 hot regions will be moved to the new machine?
          Hide
          Ted Yu added a comment -

          If the 50 hot regions are created due to region splits, load balancer is able to distinguish them from cold regions and pick both hot and cold regions for the new server.

          If the hot regions were created due to pre-split, I have code in internal repository to use region load information in achieving balance. This code is to be open-sourced.

          Show
          Ted Yu added a comment - If the 50 hot regions are created due to region splits, load balancer is able to distinguish them from cold regions and pick both hot and cold regions for the new server. If the hot regions were created due to pre-split, I have code in internal repository to use region load information in achieving balance. This code is to be open-sourced.
          Hide
          gaojinchao added a comment -

          Thanks, I am looking forward to seeing this part of the code.

          Sorry,I didn't express clearly.

          In my scenario:
          1.There is 10 nodes cluster. 1000 regions per node.
          2.We add a new machine to the cluster.
          3.Balance needs move "1000* 10/11= 909" regions to the new machine. each region server will move both 45 hot regions and 45 cold regions to the new one. in this case, all hot regions will move to this new one?

          Show
          gaojinchao added a comment - Thanks, I am looking forward to seeing this part of the code. Sorry,I didn't express clearly. In my scenario: 1.There is 10 nodes cluster. 1000 regions per node. 2.We add a new machine to the cluster. 3.Balance needs move "1000* 10/11= 909" regions to the new machine. each region server will move both 45 hot regions and 45 cold regions to the new one. in this case, all hot regions will move to this new one?
          Hide
          Ted Yu added a comment -

          If I remember correctly, this scenario might be due to your limiting the total number of active regions on each region server.
          Please consider raising the limit because 0.92 has reduced footprint of each region in Master's heap.

          If you cannot increase the limit to close to 1000 regions, please consider offlining inactive regions during each day.

          Show
          Ted Yu added a comment - If I remember correctly, this scenario might be due to your limiting the total number of active regions on each region server. Please consider raising the limit because 0.92 has reduced footprint of each region in Master's heap. If you cannot increase the limit to close to 1000 regions, please consider offlining inactive regions during each day.
          Hide
          gaojinchao added a comment -

          Yes, you are right.
          we can't increase the limit to close to 1000 regions. because another reason is when hot regions are too many, It will produce many small Hfile.
          I have a solution to deal with this case. I will delevop a new balance algorithm for our scenario.

          Show
          gaojinchao added a comment - Yes, you are right. we can't increase the limit to close to 1000 regions. because another reason is when hot regions are too many, It will produce many small Hfile. I have a solution to deal with this case. I will delevop a new balance algorithm for our scenario.
          Hide
          Ted Yu added a comment -

          HBASE-5219 tries to address Jinchao's comment above.

          Show
          Ted Yu added a comment - HBASE-5219 tries to address Jinchao's comment above.
          Hide
          stack added a comment -

          Superceded by stochasticbalancer. Resolving as implemented

          Show
          stack added a comment - Superceded by stochasticbalancer. Resolving as implemented
          stack made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Implemented [ 10 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          1031d 7h 44m 1 stack 26/Jan/14 22:47

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development