Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-614

TestDatanodeBlockScanner obtain should data-node directories directly from MiniDFSCluster

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None

      Description

      TestDatanodeBlockScanner relies on that data-node directories are listed in test.build.data, which is not true if the test run from eclipse. It shold get the directories directly from MiniDFSCluster.

      1. TestDNBlockScanner.patch
        2 kB
        Konstantin Shvachko

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #6 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/6/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #6 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/6/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #26 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/26/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #26 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/26/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #82 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/82/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #82 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/82/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #31 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/31/)
        . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #31 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/31/ ) . TestDatanodeBlockScanner obtains data directories directly from MiniHDFSCluster. Contributed by Konstantin Shvachko.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this.

        Show
        Konstantin Shvachko added a comment - I just committed this.
        Hide
        Konstantin Shvachko added a comment -

        TestBalancer timed out. Unrelated to the patch.

        Show
        Konstantin Shvachko added a comment - TestBalancer timed out. Unrelated to the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12419357/TestDNBlockScanner.patch
        against trunk revision 814047.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12419357/TestDNBlockScanner.patch against trunk revision 814047. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/22/console This message is automatically generated.
        Hide
        Konstantin Shvachko added a comment -

        Here is the patch that does it.
        I also added assertions to make sure that blocks are really corrupted. This makes the test fail rather than loop indefinitely in case the block was not corrupted.

        Show
        Konstantin Shvachko added a comment - Here is the patch that does it. I also added assertions to make sure that blocks are really corrupted. This makes the test fail rather than loop indefinitely in case the block was not corrupted.

          People

          • Assignee:
            Konstantin Shvachko
            Reporter:
            Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development