HBase
  1. HBase
  2. HBASE-8957

TestDrainingServer. testDrainingServerWithAbort fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Make it so tests run WITHOUT short circuit reads by default.
    1. 8957.txt
      6 kB
      stack
    2. 8957v2.txt
      8 kB
      stack
    3. 8957.addendum.txt
      5 kB
      stack

      Activity

      Hide
      stack added a comment -

      Resolving. This does not fail anymore.

      Show
      stack added a comment - Resolving. This does not fail anymore.
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in HBase-TRUNK #4264 (See https://builds.apache.org/job/HBase-TRUNK/4264/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504308)

      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4264 (See https://builds.apache.org/job/HBase-TRUNK/4264/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504308) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #621 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/621/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504308)

      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #621 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/621/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504308) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in hbase-0.95 #333 (See https://builds.apache.org/job/hbase-0.95/333/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504309)

      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - SUCCESS: Integrated in hbase-0.95 #333 (See https://builds.apache.org/job/hbase-0.95/333/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504309) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      FAILURE: Integrated in hbase-0.95-on-hadoop2 #183 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/183/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504309)

      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - FAILURE: Integrated in hbase-0.95-on-hadoop2 #183 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/183/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails; ADDENDUM (stack: rev 1504309) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      stack added a comment -

      Above fail is https://issues.apache.org/jira/browse/HBASE-6127 It is an outstanding issue we need to fix.

      Show
      stack added a comment - Above fail is https://issues.apache.org/jira/browse/HBASE-6127 It is an outstanding issue we need to fix.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12592865/8957.addendum.txt
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 3 new or modified tests.

      +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 lineLengths. The patch does not introduce lines longer than 100

      +1 site. The mvn site goal succeeds with this patch.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.regionserver.TestAtomicOperation

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12592865/8957.addendum.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestAtomicOperation Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6386//console This message is automatically generated.
      Hide
      stack added a comment -

      I applied it to 0.95 and trunk. Lets see how it does.

      Show
      stack added a comment - I applied it to 0.95 and trunk. Lets see how it does.
      Hide
      stack added a comment -

      Addendum that uses getOnlineRegionsLocalContext which is meant for this kind of use in tests.

      I also shut down access to the dodgy method that is only for memstore to use when up against the wall trying to figure a region to flush when globally full.

      Show
      stack added a comment - Addendum that uses getOnlineRegionsLocalContext which is meant for this kind of use in tests. I also shut down access to the dodgy method that is only for memstore to use when up against the wall trying to figure a region to flush when globally full.
      Hide
      stack added a comment -

      Ok. This was a new failure. Test was using getCopyOfOnlineRegionsSortedBySize from HRS to get list of online regions but this is unreliable since it is keyed by regionsize. When regionsizes were the same, we'd overwrite and so test would fail because never found all regions.

      Show
      stack added a comment - Ok. This was a new failure. Test was using getCopyOfOnlineRegionsSortedBySize from HRS to get list of online regions but this is unreliable since it is keyed by regionsize. When regionsizes were the same, we'd overwrite and so test would fail because never found all regions.
      Hide
      stack added a comment -

      Overnight tests turned up https://builds.apache.org/job/hbase-0.95-on-hadoop2/182/testReport/junit/org.apache.hadoop.hbase/TestDrainingServer/testDrainingServerWithAbort/ which is same as the failure above when I tried this against hadoopqa. Let me look at it and see if can do an addendum.

      Show
      stack added a comment - Overnight tests turned up https://builds.apache.org/job/hbase-0.95-on-hadoop2/182/testReport/junit/org.apache.hadoop.hbase/TestDrainingServer/testDrainingServerWithAbort/ which is same as the failure above when I tried this against hadoopqa. Let me look at it and see if can do an addendum.
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #620 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/620/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504005)

      • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #620 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/620/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504005) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      FAILURE: Integrated in hbase-0.95-on-hadoop2 #182 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/182/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504003)

      • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - FAILURE: Integrated in hbase-0.95-on-hadoop2 #182 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/182/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504003) /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      FAILURE: Integrated in HBase-TRUNK #4260 (See https://builds.apache.org/job/HBase-TRUNK/4260/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504005)

      • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
      • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
      • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #4260 (See https://builds.apache.org/job/HBase-TRUNK/4260/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504005) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      Hudson added a comment -

      FAILURE: Integrated in hbase-0.95 #331 (See https://builds.apache.org/job/hbase-0.95/331/)
      HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504003)

      • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
      • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
      • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Show
      Hudson added a comment - FAILURE: Integrated in hbase-0.95 #331 (See https://builds.apache.org/job/hbase-0.95/331/ ) HBASE-8957 TestDrainingServer. testDrainingServerWithAbort fails (stack: rev 1504003) /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
      Hide
      stack added a comment -

      I applied to trunk and 0.95 so can see how this does overnight on ec2 and trunk. Also wrote @nkeywal direct to chat about short circuit reads and if ok that it is off.

      Show
      stack added a comment - I applied to trunk and 0.95 so can see how this does overnight on ec2 and trunk. Also wrote @nkeywal direct to chat about short circuit reads and if ok that it is off.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12592668/8957v2.txt
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 6 new or modified tests.

      +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 lineLengths. The patch does not introduce lines longer than 100

      +1 site. The mvn site goal succeeds with this patch.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.security.access.TestAccessController

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12592668/8957v2.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.security.access.TestAccessController Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6365//console This message is automatically generated.
      Hide
      stack added a comment -

      Add more logging.

      Show
      stack added a comment - Add more logging.
      Hide
      stack added a comment -

      Thats an embarrassing fail.

      Show
      stack added a comment - Thats an embarrassing fail.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12592631/8957.txt
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 6 new or modified tests.

      +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 lineLengths. The patch does not introduce lines longer than 100

      +1 site. The mvn site goal succeeds with this patch.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.TestDrainingServer

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12592631/8957.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6360//console This message is automatically generated.
      Hide
      Jimmy Xiang added a comment -

      +1 on disabling it for unit tests, +1 on adding specific tests for local reads.

      Show
      Jimmy Xiang added a comment - +1 on disabling it for unit tests, +1 on adding specific tests for local reads.
      Hide
      stack added a comment -

      Nicolas Liochon You good w/ this one? Disabling short circuit reads as default because regionservers can't do hdfs ops (hadoop2 at least)?

      Show
      stack added a comment - Nicolas Liochon You good w/ this one? Disabling short circuit reads as default because regionservers can't do hdfs ops (hadoop2 at least)?
      Hide
      Jean-Daniel Cryans added a comment -

      So I'm +1 on disabling local reads. Also less spam.

      Show
      Jean-Daniel Cryans added a comment - So I'm +1 on disabling local reads. Also less spam.
      Hide
      Jean-Daniel Cryans added a comment -

      I disabled local reads on the unit tests I'm working on in HBASE-8961 and 2 of them run in half the time it used to take.

      Show
      Jean-Daniel Cryans added a comment - I disabled local reads on the unit tests I'm working on in HBASE-8961 and 2 of them run in half the time it used to take.
      Hide
      stack added a comment -

      Issue is permissions when running with short-circuit reads. Recovering of the .META. log edits fails and so .META. is stuck in RIT till the test times out.

      In the logs you see:

      org.apache.hadoop.security.AccessControlException: Can't continue with getBlockLocalPathInfo() authorization. The user jenkins.hfs.4 is not allowed to call getBlockLocalPathInfo
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.checkBlockLocalPathAccess(DataNode.java:1013)
      
      

      The 'jenkins.hfs.4' is the RS trying to replay edits. It is failing because only the user 'jenkins' has perms to do short circuit reading.

      The '.hfs.4' suffix is added in HBaseTestingUtility when we start up a little hbase cluster... it is added per regionserver.

      So, we could reach back into the little mnincluster and IFF shortcircuit reads are enabled, then we have all RS use same user accessing hdfs.

      But I think this will break tests. So I went the other route and disabled shortcircuit reading as the default for mini clusters. Quiet a few tests disable it already.

      We probably need to add an explicit shortcircuit tests.

       30 Patch that disables shortcircuit reading as default when running unit tests
       29 because when a minicluster, not all regionservers have permissions to do
       28 short circuit reading.
       27
       26 May slow tests some.
       25
       24
       23 M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
       22   Unrelated: Shorten log message and add some context to explain a mysterious log.
       21
       20 M hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
       19   Unrelated: Shorten a log message.
       18
       17
       16 M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
       15   Unrelated refactoring of a getMetaWAL method.  This getMetaWAL is actually kinda
       14   wrong in that it will do the creation if does not already exist which makes for
       13   interesting parents:
       12
       11     2013-07-15 12:04:04,245 INFO  [RS_OPEN_META-juno:36201-0MetaLogRoller] regionserver.LogRoller(119): LogRoller exiting.
       10
        9   An OPEN executor has the meta as its parent it seems which probably means
        8   the executor has an occupied thread, a thread that won't go down.  TODO.
        7
        6   Also fix name of metalogroller thread so it is like others.
        5
        4 M hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        3   Main fix changing short circuit read to default off.
        2
        1 M hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
        0   Unrelated... add some sleep so we don't spew logs.
      
      Show
      stack added a comment - Issue is permissions when running with short-circuit reads. Recovering of the .META. log edits fails and so .META. is stuck in RIT till the test times out. In the logs you see: org.apache.hadoop.security.AccessControlException: Can't continue with getBlockLocalPathInfo() authorization. The user jenkins.hfs.4 is not allowed to call getBlockLocalPathInfo at org.apache.hadoop.hdfs.server.datanode.DataNode.checkBlockLocalPathAccess(DataNode.java:1013) The 'jenkins.hfs.4' is the RS trying to replay edits. It is failing because only the user 'jenkins' has perms to do short circuit reading. The '.hfs.4' suffix is added in HBaseTestingUtility when we start up a little hbase cluster... it is added per regionserver. So, we could reach back into the little mnincluster and IFF shortcircuit reads are enabled, then we have all RS use same user accessing hdfs. But I think this will break tests. So I went the other route and disabled shortcircuit reading as the default for mini clusters. Quiet a few tests disable it already. We probably need to add an explicit shortcircuit tests. 30 Patch that disables shortcircuit reading as default when running unit tests 29 because when a minicluster, not all regionservers have permissions to do 28 short circuit reading. 27 26 May slow tests some. 25 24 23 M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 22 Unrelated: Shorten log message and add some context to explain a mysterious log. 21 20 M hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 19 Unrelated: Shorten a log message. 18 17 16 M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 15 Unrelated refactoring of a getMetaWAL method. This getMetaWAL is actually kinda 14 wrong in that it will do the creation if does not already exist which makes for 13 interesting parents: 12 11 2013-07-15 12:04:04,245 INFO [RS_OPEN_META-juno:36201-0MetaLogRoller] regionserver.LogRoller(119): LogRoller exiting. 10 9 An OPEN executor has the meta as its parent it seems which probably means 8 the executor has an occupied thread, a thread that won't go down. TODO. 7 6 Also fix name of metalogroller thread so it is like others. 5 4 M hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 3 Main fix changing short circuit read to default off. 2 1 M hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java 0 Unrelated... add some sleep so we don't spew logs.
      Hide
      stack added a comment -

      It looks to be a perms problem when doing dfs issues. The jenkins user is cleared to do short circuit reading but then regionservers are running as jenkins.hfs.1, jenkins.hfs.2, etc.

      Wondering why we usually pass this test, looking at a 'successful' build, it looks to me like it is not running at all:

      https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/617/testReport/org.apache.hadoop.hbase/TestDrainingServer/

      The times given for the tests are suspiciously short.

      Show
      stack added a comment - It looks to be a perms problem when doing dfs issues. The jenkins user is cleared to do short circuit reading but then regionservers are running as jenkins.hfs.1, jenkins.hfs.2, etc. Wondering why we usually pass this test, looking at a 'successful' build, it looks to me like it is not running at all: https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/617/testReport/org.apache.hadoop.hbase/TestDrainingServer/ The times given for the tests are suspiciously short.
      Show
      stack added a comment - And here http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/631/testReport/org.apache.hadoop.hbase/TestDrainingServer/testDrainingServerWithAbort/

        People

        • Assignee:
          stack
          Reporter:
          stack
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development