Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, hbase-10070
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      One of the tests introduced in HBASE-10351 seems to be flaky. The LB cannot compute the fully assignment plan in time when there are racks and region replicas for the test, so it is failing sometimes.

      We can reduce the computation amount, and increase the LB runtime to make the test stable.

      1. hbase-10661_v1.patch
        2 kB
        Enis Soztutar
      2. hbase-10661_v2.patch
        17 kB
        Enis Soztutar
      3. hbase-10661_v3.patch
        19 kB
        Enis Soztutar
      4. hbase-10661_v4.patch
        19 kB
        Enis Soztutar
      5. hbase-10661_v5.patch
        21 kB
        Enis Soztutar
      6. hbase-10661_v6.patch
        24 kB
        Enis Soztutar
      7. 0036-HBASE-10661-TestStochasticLoadBalancer.testRegionRep.patch
        24 kB
        Enis Soztutar

        Activity

        Enis Soztutar created issue -
        Hide
        Enis Soztutar added a comment -

        Simple patch. Also gets rid of unused mock class.

        Show
        Enis Soztutar added a comment - Simple patch. Also gets rid of unused mock class.
        Enis Soztutar made changes -
        Field Original Value New Value
        Attachment hbase-10661_v1.patch [ 12632429 ]
        Hide
        Devaraj Das added a comment -

        +1

        Show
        Devaraj Das added a comment - +1
        Hide
        Enis Soztutar added a comment -

        Attaching a v2 patch, since we still see some failures after v1 patch.

        This patch adds another candidate generator which is trying to move away the regions which are co-located in the same rack. This will help with the test as well as the production cases because the LB is expected to converse sooner.

        Show
        Enis Soztutar added a comment - Attaching a v2 patch, since we still see some failures after v1 patch. This patch adds another candidate generator which is trying to move away the regions which are co-located in the same rack. This will help with the test as well as the production cases because the LB is expected to converse sooner.
        Enis Soztutar made changes -
        Attachment hbase-10661_v2.patch [ 12633291 ]
        Hide
        Enis Soztutar added a comment -

        Attaching v3 patch. v2 dropped some changes because of a mis-rebase on my part. v3 is v1 + v2.

        Show
        Enis Soztutar added a comment - Attaching v3 patch. v2 dropped some changes because of a mis-rebase on my part. v3 is v1 + v2.
        Enis Soztutar made changes -
        Attachment hbase-10661_v3.patch [ 12633467 ]
        Hide
        Enis Soztutar added a comment -

        v4 patch with increased timeouts. I'll let it run a couple more days in our rig to see the results.

        Show
        Enis Soztutar added a comment - v4 patch with increased timeouts. I'll let it run a couple more days in our rig to see the results.
        Enis Soztutar made changes -
        Attachment hbase-10661_v4.patch [ 12634092 ]
        Hide
        Enis Soztutar added a comment -

        v5 should be good enough to commit. We've been running the tests on multiple platforms, and the tests seems stable enough.

        Show
        Enis Soztutar added a comment - v5 should be good enough to commit. We've been running the tests on multiple platforms, and the tests seems stable enough.
        Enis Soztutar made changes -
        Attachment hbase-10661_v5.patch [ 12639401 ]
        Hide
        Enis Soztutar added a comment -

        Devaraj Das can you review the latest version as well. The tests have been more stable after this for some time. v6 should make it even more so.

        Show
        Enis Soztutar added a comment - Devaraj Das can you review the latest version as well. The tests have been more stable after this for some time. v6 should make it even more so.
        Enis Soztutar made changes -
        Attachment hbase-10661_v6.patch [ 12642777 ]
        Hide
        Devaraj Das added a comment -

        +1

        Show
        Devaraj Das added a comment - +1
        Hide
        Enis Soztutar added a comment -

        Thanks Devaraj. I've committed this to branch.

        Show
        Enis Soztutar added a comment - Thanks Devaraj. I've committed this to branch.
        Enis Soztutar made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Enis Soztutar made changes -
        Hide
        Enis Soztutar added a comment -

        Attaching rebased patch for master that is committed

        Show
        Enis Soztutar added a comment - Attaching rebased patch for master that is committed
        Hide
        Enis Soztutar added a comment -

        Committed to master as part of hbase-10070 branch merge

        Show
        Enis Soztutar added a comment - Committed to master as part of hbase-10070 branch merge
        Enis Soztutar made changes -
        Fix Version/s 0.99.0 [ 12325675 ]
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5245 (See https://builds.apache.org/job/HBase-TRUNK/5245/)
        HBASE-10661 TestStochasticLoadBalancer.testRegionReplicationOnMidClusterWithRacks() is flaky (enis: rev d84c863525e9fb1685f3e1c8afe05702454ca217)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5245 (See https://builds.apache.org/job/HBase-TRUNK/5245/ ) HBASE-10661 TestStochasticLoadBalancer.testRegionReplicationOnMidClusterWithRacks() is flaky (enis: rev d84c863525e9fb1685f3e1c8afe05702454ca217) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        66d 15h 10m 1 Enis Soztutar 09/May/14 18:14
        Resolved Resolved Closed Closed
        288d 5h 21m 1 Enis Soztutar 21/Feb/15 23:35

          People

          • Assignee:
            Enis Soztutar
            Reporter:
            Enis Soztutar
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development