Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-614

TestDatanodeBlockScanner obtain should data-node directories directly from MiniDFSCluster

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None

      Description

      TestDatanodeBlockScanner relies on that data-node directories are listed in test.build.data, which is not true if the test run from eclipse. It shold get the directories directly from MiniDFSCluster.

      1. TestDNBlockScanner.patch
        2 kB
        Konstantin Shvachko

        Activity

        Hide
        shv Konstantin Shvachko added a comment -

        Here is the patch that does it.
        I also added assertions to make sure that blocks are really corrupted. This makes the test fail rather than loop indefinitely in case the block was not corrupted.

        Show
        shv Konstantin Shvachko added a comment - Here is the patch that does it. I also added assertions to make sure that blocks are really corrupted. This makes the test fail rather than loop indefinitely in case the block was not corrupted.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12419357/TestDNBlockScanner.patch
        against trunk revision 814047.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12419357/TestDNBlockScanner.patch against trunk revision 814047. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/console This message is automatically generated.
        Hide
        shv Konstantin Shvachko added a comment -

        TestBalancer timed out. Unrelated to the patch.

        Show
        shv Konstantin Shvachko added a comment - TestBalancer timed out. Unrelated to the patch.
        Hide
        shv Konstantin Shvachko added a comment -

        I just committed this.

        Show
        shv Konstantin Shvachko added a comment - I just committed this.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #31 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/31/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #31 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/31/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #82 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/82/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #82 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/82/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.
        Hide
        hudson Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #26 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/26/)

        Show
        hudson Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #26 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/26/ )
        Hide
        hudson Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #6 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/6/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        hudson Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #6 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/6/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

          People

          • Assignee:
            shv Konstantin Shvachko
            Reporter:
            shv Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development