HBase
  1. HBase
  2. HBASE-5760

Unit tests should write only under /target

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.95.2
    • Fix Version/s: 0.95.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Some of the unit test runs result in files under $hbase_home/test, $hbase_home/build, or $hbase_home/. We should ensure that all tests use target as their data location.

        Activity

        Hide
        stack added a comment -

        +1

        The /build are because of hdfs IIRC.

        The /test one is annoying. I think that from mr.

        Thanks Enis.

        Show
        stack added a comment - +1 The /build are because of hdfs IIRC. The /test one is annoying. I think that from mr. Thanks Enis.
        Hide
        stack added a comment -

        It does seem like something Mikhail would have fixed in HBASE-5747. I've not checked.

        Show
        stack added a comment - It does seem like something Mikhail would have fixed in HBASE-5747 . I've not checked.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4692/
        -----------------------------------------------------------

        (Updated 2012-04-10 20:34:07.296907)

        Review request for hbase.

        Summary
        -------

        Some of the unit test runs result in files under $hbase_home/test, $hbase_home/build, or $hbase_home/. We should ensure that all tests use target as their data location.

        This addresses bug HBASE-5760.
        https://issues.apache.org/jira/browse/HBASE-5760

        Diffs


        src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 3f239e0
        src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 32f0cc3
        src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java 60f21c0
        src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java 002d611
        src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java a6bcbac

        Diff: https://reviews.apache.org/r/4692/diff

        Testing
        -------

        Results :

        Failed tests: testKeyUpdate(org.apache.hadoop.hbase.security.token.TestZKSecretWatcher): expected:<AuthenticationKey[ id=2, expiration=9223372036854775807 ]> but was:<AuthenticationKey[ id=1, expiration=1334028188780 ]>

        Tests in error:
        testDelayedRpcImmediateReturnValue(org.apache.hadoop.hbase.ipc.TestDelayedRpc): Call to /127.0.0.1:65022 failed on socket timeout exception: java.net.SocketTimeoutException: 1000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:65039 remote=/127.0.0.1:65022]
        testTralingGarbageCorruptionFileSkipErrorsPasses(org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit): All datanodes 127.0.0.1:52012 are bad. Aborting...
        org.apache.hadoop.hbase.security.access.TestAccessControlFilter: Shutting down
        org.apache.hadoop.hbase.security.access.TestAccessController: Shutting down
        org.apache.hadoop.hbase.security.access.TestTablePermissions: Shutting down
        org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher: Shutting down
        org.apache.hadoop.hbase.security.token.TestTokenAuthentication: Shutting down

        Tests run: 900, Failures: 1, Errors: 7, Skipped: 9

        I think the ZKSecretWatcher is unrelated. After the tests, there is no residual test files lying around.

        Thanks,

        enis

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4692/ ----------------------------------------------------------- (Updated 2012-04-10 20:34:07.296907) Review request for hbase. Summary ------- Some of the unit test runs result in files under $hbase_home/test, $hbase_home/build, or $hbase_home/. We should ensure that all tests use target as their data location. This addresses bug HBASE-5760 . https://issues.apache.org/jira/browse/HBASE-5760 Diffs src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 3f239e0 src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 32f0cc3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java 60f21c0 src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java 002d611 src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java a6bcbac Diff: https://reviews.apache.org/r/4692/diff Testing ------- Results : Failed tests: testKeyUpdate(org.apache.hadoop.hbase.security.token.TestZKSecretWatcher): expected:<AuthenticationKey[ id=2, expiration=9223372036854775807 ]> but was:<AuthenticationKey[ id=1, expiration=1334028188780 ]> Tests in error: testDelayedRpcImmediateReturnValue(org.apache.hadoop.hbase.ipc.TestDelayedRpc): Call to /127.0.0.1:65022 failed on socket timeout exception: java.net.SocketTimeoutException: 1000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel [connected local=/127.0.0.1:65039 remote=/127.0.0.1:65022] testTralingGarbageCorruptionFileSkipErrorsPasses(org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit): All datanodes 127.0.0.1:52012 are bad. Aborting... org.apache.hadoop.hbase.security.access.TestAccessControlFilter: Shutting down org.apache.hadoop.hbase.security.access.TestAccessController: Shutting down org.apache.hadoop.hbase.security.access.TestTablePermissions: Shutting down org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher: Shutting down org.apache.hadoop.hbase.security.token.TestTokenAuthentication: Shutting down Tests run: 900, Failures: 1, Errors: 7, Skipped: 9 I think the ZKSecretWatcher is unrelated. After the tests, there is no residual test files lying around. Thanks, enis
        Hide
        Enis Soztutar added a comment -

        Attaching the patch from the review. Thanks Stack for the review. I'll take a look at HBASE-5747 shortly.

        Show
        Enis Soztutar added a comment - Attaching the patch from the review. Thanks Stack for the review. I'll take a look at HBASE-5747 shortly.
        Hide
        Enis Soztutar added a comment -

        The downstream patch in HBASE-5747 seems not much correlated with this patch. The biggest change in this is to fix up the MiniMRCluster to use the proper mapred.local.dir, which should also help HBASE-5747. Other tests are not there at 0.89-fb.

        Show
        Enis Soztutar added a comment - The downstream patch in HBASE-5747 seems not much correlated with this patch. The biggest change in this is to fix up the MiniMRCluster to use the proper mapred.local.dir, which should also help HBASE-5747 . Other tests are not there at 0.89-fb.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522164/HBASE-5760_v1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 18 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522164/HBASE-5760_v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1468//console This message is automatically generated.
        Hide
        stack added a comment -

        Committed to trunk. Thanks for the patch Enis.

        Show
        stack added a comment - Committed to trunk. Thanks for the patch Enis.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2740 (See https://builds.apache.org/job/HBase-TRUNK/2740/)
        HBASE-5760 Unit tests should write only under /target (Revision 1312043)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2740 (See https://builds.apache.org/job/HBase-TRUNK/2740/ ) HBASE-5760 Unit tests should write only under /target (Revision 1312043) Result = SUCCESS stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #167 (See https://builds.apache.org/job/HBase-TRUNK-security/167/)
        HBASE-5760 Unit tests should write only under /target (Revision 1312043)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #167 (See https://builds.apache.org/job/HBase-TRUNK-security/167/ ) HBASE-5760 Unit tests should write only under /target (Revision 1312043) Result = SUCCESS stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-security #241 (See https://builds.apache.org/job/HBase-0.94-security/241/)
        HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (enis: rev 1509572)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-security #241 (See https://builds.apache.org/job/HBase-0.94-security/241/ ) HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (enis: rev 1509572) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94 #1089 (See https://builds.apache.org/job/HBase-0.94/1089/)
        HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (enis: rev 1509572)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94 #1089 (See https://builds.apache.org/job/HBase-0.94/1089/ ) HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (enis: rev 1509572) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94-security #242 (See https://builds.apache.org/job/HBase-0.94-security/242/)
        HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (addendum patch to fix Hadoop2 build) (enis: rev 1509873)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94-security #242 (See https://builds.apache.org/job/HBase-0.94-security/242/ ) HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (addendum patch to fix Hadoop2 build) (enis: rev 1509873) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94 #1091 (See https://builds.apache.org/job/HBase-0.94/1091/)
        HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (addendum patch to fix Hadoop2 build) (enis: rev 1509873)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94 #1091 (See https://builds.apache.org/job/HBase-0.94/1091/ ) HBASE-9075 [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 (addendum patch to fix Hadoop2 build) (enis: rev 1509873) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java

          People

          • Assignee:
            Enis Soztutar
            Reporter:
            Enis Soztutar
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development