Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestDiskError.testLocalDirs() fails with FileNotFoundException

      1. hdfs-1101-y20s-v1-d1.patch
        1 kB
        Luke Lu
      2. hdfs-1101-y20s-v1.patch
        0.7 kB
        Luke Lu
      3. H1101-0.patch
        1 kB
        Chris Douglas
      4. TestDiskErrorLocal.patch
        2 kB
        Konstantin Shvachko

        Activity

        Hide
        Luke Lu added a comment -

        Somehow missed half of the fixes for the y20s patch.

        Show
        Luke Lu added a comment - Somehow missed half of the fixes for the y20s patch.
        Hide
        Luke Lu added a comment -

        TestDiskError would fail in testLocalDirs due to "incompatible namespaceIDs" exception on subsequent runs in y20s branch. The attached patch fix the test cluster initialization a la trunk.

        Show
        Luke Lu added a comment - TestDiskError would fail in testLocalDirs due to "incompatible namespaceIDs" exception on subsequent runs in y20s branch. The attached patch fix the test cluster initialization a la trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442341/H1101-0.patch
        against trunk revision 937691.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442341/H1101-0.patch against trunk revision 937691. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/325/console This message is automatically generated.
        Hide
        Jakob Homan added a comment -

        I've committed this. Thanks Chris and Konstantin! Resolving as fixed.

        Show
        Jakob Homan added a comment - I've committed this. Thanks Chris and Konstantin! Resolving as fixed.
        Hide
        Jakob Homan added a comment -

        +1 on H1101-0.patch

        Show
        Jakob Homan added a comment - +1 on H1101-0.patch
        Hide
        Chris Douglas added a comment -

        Resubmitting without MiniDFSCluster changes.

        Show
        Chris Douglas added a comment - Resubmitting without MiniDFSCluster changes.
        Hide
        Jakob Homan added a comment -

        This patch no longer cleanly applies after HDFS-909. Also, I'm not getting the test failure anymore. That patch had two of the changes to MiniDFSCluster that are included in this patch.

        Show
        Jakob Homan added a comment - This patch no longer cleanly applies after HDFS-909 . Also, I'm not getting the test failure anymore. That patch had two of the changes to MiniDFSCluster that are included in this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442342/H1101-1.patch
        against trunk revision 936024.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442342/H1101-1.patch against trunk revision 936024. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/320/console This message is automatically generated.
        Hide
        Konstantin Shvachko added a comment -

        I agree, this looks better. Thanks.
        +1 for the patch.

        Show
        Konstantin Shvachko added a comment - I agree, this looks better. Thanks. +1 for the patch.
        Hide
        Chris Douglas added a comment -

        Forgot to include Konstantin's javac warning fixes

        Show
        Chris Douglas added a comment - Forgot to include Konstantin's javac warning fixes
        Hide
        Chris Douglas added a comment -

        I'm sorry, I missed this in review.

        Though the current patch works for this case with only 1 datanode, pulling it from the DataNode is closer to the intent of the test and doesn't modify MiniDFSCluster.

        Show
        Chris Douglas added a comment - I'm sorry, I missed this in review. Though the current patch works for this case with only 1 datanode, pulling it from the DataNode is closer to the intent of the test and doesn't modify MiniDFSCluster.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442252/TestDiskErrorLocal.patch
        against trunk revision 935778.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442252/TestDiskErrorLocal.patch against trunk revision 935778. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/319/console This message is automatically generated.
        Hide
        Konstantin Shvachko added a comment -

        MiniDFSCluster overrides the data-node storage directories while the original config is till pointing to the default values.
        Therefore the directory cannot be found.
        The patch fixes the problem and two java warnings in MiniDFSCluster.

        Show
        Konstantin Shvachko added a comment - MiniDFSCluster overrides the data-node storage directories while the original config is till pointing to the default values. Therefore the directory cannot be found. The patch fixes the problem and two java warnings in MiniDFSCluster.
        Hide
        Konstantin Shvachko added a comment -

        Looks like this was introduced by HDFS-997.

        Show
        Konstantin Shvachko added a comment - Looks like this was introduced by HDFS-997 .
        Hide
        Konstantin Shvachko added a comment -

        The exceptino and the stack trace are here:

        Testcase: testLocalDirs took 1.056 sec
        	Caused an ERROR
        File file:/tmp/hadoop-shv/dfs/data does not exist.
        java.io.FileNotFoundException: File file:/tmp/hadoop-shv/dfs/data does not exist.
        	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420)
        	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246)
        	at org.apache.hadoop.hdfs.server.datanode.TestDiskError.testLocalDirs(TestDiskError.java:172)
        
        Show
        Konstantin Shvachko added a comment - The exceptino and the stack trace are here: Testcase: testLocalDirs took 1.056 sec Caused an ERROR File file:/tmp/hadoop-shv/dfs/data does not exist. java.io.FileNotFoundException: File file:/tmp/hadoop-shv/dfs/data does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246) at org.apache.hadoop.hdfs.server.datanode.TestDiskError.testLocalDirs(TestDiskError.java:172)

          People

          • Assignee:
            Chris Douglas
            Reporter:
            Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development