Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9095

AssignmentManager's handleRegion should respect the single threaded nature of the processing

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.2
    • Component/s: Region Assignment
    • Labels:
      None

      Description

      While debugging a case where a region was getting opened on a RegionServer and then closed soon after (and then never re-opened anywhere thereafter), it seemed like the processing in handleRegion to do with deletion of ZK nodes should be non-asynchronous. This achieves two things:
      1. The synchronous deletion prevents more than one processing on the same event data twice. Assuming that we do get more than one notification (on let's say, region OPENED event), the subsequent processing(s) in handleRegion for the same znode would end up with a zookeeper node not found exception. The return value of the data read would be null and that's already handled. If it is asynchronous, it leads to issues like - master opens a region on a certain RegionServer and soon after it sends that RegionServer a close for the same region, and then the znode is deleted.

      2. The deletion is currently handled in an executor service. This is problematic since by design the events for a given region should be processed in order. By delegating a part of the processing to executor service we are somewhat violating this contract since there is no guarantee of the ordering in the executor service executions...

      Thanks to Jeffrey Zhong and Enis Soztutar for the discussions on this issue.

      1. 9095-1.txt
        3 kB
        stack
      2. 9095-1.txt
        3 kB
        Devaraj Das
      3. 9095-1.txt
        3 kB
        Devaraj Das
      4. 9095-1 (1).txt
        3 kB
        stack
      5. 9095-2.txt
        17 kB
        Devaraj Das

        Activity

        Hide
        devaraj Devaraj Das added a comment -

        Should fix the issue.

        Show
        devaraj Devaraj Das added a comment - Should fix the issue.
        Hide
        devaraj Devaraj Das added a comment -

        Reattaching again since hadoopqa didn't pick it up the last time.

        Show
        devaraj Devaraj Das added a comment - Reattaching again since hadoopqa didn't pick it up the last time.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12595621/9095-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595621/9095-1.txt against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6573//console This message is automatically generated.
        Hide
        jxiang Jimmy Xiang added a comment -

        Now, since we have multiple threads to process assignment events, I think it is good to run these handlers synchronously.

        +1 for the patch

        Show
        jxiang Jimmy Xiang added a comment - Now, since we have multiple threads to process assignment events, I think it is good to run these handlers synchronously. +1 for the patch
        Hide
        enis Enis Soztutar added a comment -

        Agreed. If we find any slowness in assignment, we can address them later.

        Show
        enis Enis Soztutar added a comment - Agreed. If we find any slowness in assignment, we can address them later.
        Hide
        enis Enis Soztutar added a comment -

        +1

        Show
        enis Enis Soztutar added a comment - +1
        Hide
        stack stack added a comment -

        Please rerun this patch a few times before committing. We got a zombie above. I've not seen that in a long time. Let me rerun for you now.

        Show
        stack stack added a comment - Please rerun this patch a few times before committing. We got a zombie above. I've not seen that in a long time. Let me rerun for you now.
        Hide
        stack stack added a comment -

        Since these handlers no longer run in executors, are there executors we start in the RS that are now unused? If so, should these be removed as part of this patch? Otherwise, nice one lads.

        Show
        stack stack added a comment - Since these handlers no longer run in executors, are there executors we start in the RS that are now unused? If so, should these be removed as part of this patch? Otherwise, nice one lads.
        Hide
        devaraj Devaraj Das added a comment -

        Thanks folks for the reviews. stack the failure is a real one. I can reproduce it.. The problem is that some tests rely on the executor service as a way to register listeners, and the AM executes the listener as part of running the handler code within the executor. Removing the executor invocation makes those tests hang (for example, TestZKBasedOpenCloseRegion#testCloseRegion). I am trying to see how to make such tests work.

        Show
        devaraj Devaraj Das added a comment - Thanks folks for the reviews. stack the failure is a real one. I can reproduce it.. The problem is that some tests rely on the executor service as a way to register listeners, and the AM executes the listener as part of running the handler code within the executor. Removing the executor invocation makes those tests hang (for example, TestZKBasedOpenCloseRegion#testCloseRegion). I am trying to see how to make such tests work.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12595700/9095-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595700/9095-1.txt against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6585//console This message is automatically generated.
        Hide
        stack stack added a comment -

        TestZKInterProcessReadWriteLock is a 'flakey'. See HBASE-7980. Let me rerun the patch.

        Show
        stack stack added a comment - TestZKInterProcessReadWriteLock is a 'flakey'. See HBASE-7980 . Let me rerun the patch.
        Hide
        stack stack added a comment -

        Devaraj Das Pardon me. I missed your comment that the zombies are reproducible (I also missed that it happend second time through). I'll leave this issue alone then (Hurray for the zombie detector!)

        Show
        stack stack added a comment - Devaraj Das Pardon me. I missed your comment that the zombies are reproducible (I also missed that it happend second time through). I'll leave this issue alone then (Hurray for the zombie detector!)
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12595765/9095-1%20%281%29.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 site. The patch appears to cause mvn site goal to fail.

        -1 core tests. The patch failed these unit tests:

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595765/9095-1%20%281%29.txt against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6594//console This message is automatically generated.
        Hide
        devaraj Devaraj Das added a comment -

        I had to do some more changes to have the tests work (TestMaster and TestTestZKBasedOpenCloseRegion). I was trying to make the tests work with Mockito/spy but it didn't work and at some point I gave up. This approach is not as elegant but works.

        Show
        devaraj Devaraj Das added a comment - I had to do some more changes to have the tests work (TestMaster and TestTestZKBasedOpenCloseRegion). I was trying to make the tests work with Mockito/spy but it didn't work and at some point I gave up. This approach is not as elegant but works.
        Hide
        devaraj Devaraj Das added a comment -

        Since these handlers no longer run in executors, are there executors we start in the RS that are now unused?

        Not sure on this. The executor service is used during the master recovery (look at AM.processRegionsInTransition) and I am not sure I want to touch that part in this patch...

        Show
        devaraj Devaraj Das added a comment - Since these handlers no longer run in executors, are there executors we start in the RS that are now unused? Not sure on this. The executor service is used during the master recovery (look at AM.processRegionsInTransition) and I am not sure I want to touch that part in this patch...
        Hide
        jxiang Jimmy Xiang added a comment -

        Currently, the executor service is still needed. We can get rid of it later on when we remove the timeout monitor.

        Show
        jxiang Jimmy Xiang added a comment - Currently, the executor service is still needed. We can get rid of it later on when we remove the timeout monitor.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12596163/9095-2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596163/9095-2.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6605//console This message is automatically generated.
        Hide
        ram_krish ramkrishna.s.vasudevan added a comment -

        Ok to do this in 0.94 also to avoid many of the concurrency issues? Need to see the code to see how much it helps though?
        What Jimmy Xiang, Jeffrey Zhong feel?

        Show
        ram_krish ramkrishna.s.vasudevan added a comment - Ok to do this in 0.94 also to avoid many of the concurrency issues? Need to see the code to see how much it helps though? What Jimmy Xiang , Jeffrey Zhong feel?
        Hide
        jxiang Jimmy Xiang added a comment -

        For 0.94, just one thread is used to process assignment events. Will this slow down assignments?

        For trunk, it is not a problem since we have up to 20 zk event workers by default.

        Show
        jxiang Jimmy Xiang added a comment - For 0.94, just one thread is used to process assignment events. Will this slow down assignments? For trunk, it is not a problem since we have up to 20 zk event workers by default.
        Hide
        devaraj Devaraj Das added a comment -

        Although a couple of you gave +1s before, mind taking a look at the patch once to see if it still is okay (main changes are to do with the unit test fixes).

        Show
        devaraj Devaraj Das added a comment - Although a couple of you gave +1s before, mind taking a look at the patch once to see if it still is okay (main changes are to do with the unit test fixes).
        Hide
        jxiang Jimmy Xiang added a comment -

        +1. Probably we can change how the unit test does verifications later on, so that we don't need the special counter/tracker.

        Show
        jxiang Jimmy Xiang added a comment - +1. Probably we can change how the unit test does verifications later on, so that we don't need the special counter/tracker.
        Hide
        devaraj Devaraj Das added a comment -

        Thanks Jimmy Xiang. I'll commit the patch in an hour or so unless I hear any objection.

        Show
        devaraj Devaraj Das added a comment - Thanks Jimmy Xiang . I'll commit the patch in an hour or so unless I hear any objection.
        Hide
        devaraj Devaraj Das added a comment -

        Committed to 0.95 and trunk.

        Show
        devaraj Devaraj Das added a comment - Committed to 0.95 and trunk.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in hbase-0.95-on-hadoop2 #220 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/220/)
        HBASE-9095. AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510800)

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in hbase-0.95-on-hadoop2 #220 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/220/ ) HBASE-9095 . AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510800) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #654 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/654/)
        HBASE-9095. AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510799)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #654 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/654/ ) HBASE-9095 . AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510799) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #4345 (See https://builds.apache.org/job/HBase-TRUNK/4345/)
        HBASE-9095. AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510799)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #4345 (See https://builds.apache.org/job/HBase-TRUNK/4345/ ) HBASE-9095 . AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510799) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in hbase-0.95 #406 (See https://builds.apache.org/job/hbase-0.95/406/)
        HBASE-9095. AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510800)

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in hbase-0.95 #406 (See https://builds.apache.org/job/hbase-0.95/406/ ) HBASE-9095 . AssignmentManager's handleRegion should respect the single threaded nature of the processing (ddas: rev 1510800) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java

          People

          • Assignee:
            devaraj Devaraj Das
            Reporter:
            devaraj Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development