Hadoop Common
  1. Hadoop Common
  2. HADOOP-9067

provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha, 0.23.7
    • Component/s: None
    • Labels:
      None

      Description

      this method is not covered by the existing unit tests. Provide a test to cover it.

      1. HADOOP-9067.patch
        5 kB
        Ivan A. Veselovsky
      2. HADOOP-9067--b.patch
        5 kB
        Ivan A. Veselovsky

        Activity

        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554342/HADOOP-9067.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554342/HADOOP-9067.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//console This message is automatically generated.
        Hide
        Ivan A. Veselovsky added a comment -

        the new test sligtly improved to cover more methods from LocalFileSystem.

        Show
        Ivan A. Veselovsky added a comment - the new test sligtly improved to cover more methods from LocalFileSystem.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554344/HADOOP-9067--b.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554344/HADOOP-9067--b.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        The patch is a little old (sorry about taking so long to review it) but it still applies and runs just fine on trunk and branch-0.23. I kicked Jenkins again to be sure everything else comes out OK. If it does I am +1 on this patch. I'll check it in once Jenkins finishes.

        Show
        Robert Joseph Evans added a comment - The patch is a little old (sorry about taking so long to review it) but it still applies and runs just fine on trunk and branch-0.23. I kicked Jenkins again to be sure everything else comes out OK. If it does I am +1 on this patch. I'll check it in once Jenkins finishes.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554344/HADOOP-9067--b.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2134//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2134//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554344/HADOOP-9067--b.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2134//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2134//console This message is automatically generated.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3312 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3312/)
        HADOOP-9067. provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3312 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3312/ ) HADOOP-9067 . provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Hide
        Robert Joseph Evans added a comment -

        Thanks Ivan, I put this into trunk, branch-2, and branch-0.23.

        Show
        Robert Joseph Evans added a comment - Thanks Ivan, I put this into trunk, branch-2, and branch-0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #115 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/115/)
        HADOOP-9067. provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #115 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/115/ ) HADOOP-9067 . provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/)
        svn merge -c 1441628 FIXES: HADOOP-9067. provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441638)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441638
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/ ) svn merge -c 1441628 FIXES: HADOOP-9067 . provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441638) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441638 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1304 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1304/)
        HADOOP-9067. provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1304 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1304/ ) HADOOP-9067 . provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1332 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1332/)
        HADOOP-9067. provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1332 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1332/ ) HADOOP-9067 . provide test for LocalFileSystem.reportChecksumFailure (Ivan A. Veselovsky via bobby) (Revision 1441628) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441628 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
        Hide
        Eric Yang added a comment -

        This patch is triggering test case failure on trunk:

        Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.863 sec <<< FAILURE!
        testReportChecksumFailure(org.apache.hadoop.fs.TestLocalFileSystem) Time elapsed: 42 sec <<< FAILURE!
        java.lang.AssertionError:
        at org.junit.Assert.fail(Assert.java:91)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at org.apache.hadoop.fs.TestLocalFileSystem.testReportChecksumFailure(TestLocalFileSystem.java:327)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
        at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
        at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

        Show
        Eric Yang added a comment - This patch is triggering test case failure on trunk: Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.863 sec <<< FAILURE! testReportChecksumFailure(org.apache.hadoop.fs.TestLocalFileSystem) Time elapsed: 42 sec <<< FAILURE! java.lang.AssertionError: at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.hadoop.fs.TestLocalFileSystem.testReportChecksumFailure(TestLocalFileSystem.java:327) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
        Hide
        Ivan A. Veselovsky added a comment -

        Hi, Eric,
        It looks like I cannot reproduce this failure.
        We have hadoop builds running on regular basis, and this test never fails.
        Also it passes in Apache builds, e.g. https://builds.apache.org/view/Hadoop/job/Hadoop-Common-trunk/689/consoleFull .
        Can you please provide more info on this failure: what OS/java/maven options you use?
        Also full log in debug level may be helpful.

        Show
        Ivan A. Veselovsky added a comment - Hi, Eric, It looks like I cannot reproduce this failure. We have hadoop builds running on regular basis, and this test never fails. Also it passes in Apache builds, e.g. https://builds.apache.org/view/Hadoop/job/Hadoop-Common-trunk/689/consoleFull . Can you please provide more info on this failure: what OS/java/maven options you use? Also full log in debug level may be helpful.
        Hide
        Eric Yang added a comment -

        I tested this on Mac 10.8.2 and Java 1.6.0_37, which is probably no longer supported?

        Show
        Eric Yang added a comment - I tested this on Mac 10.8.2 and Java 1.6.0_37, which is probably no longer supported?
        Hide
        Ivan A. Veselovsky added a comment -

        Hi, Eric,
        i found the reason of the failure: o.a.h.fs.DF#getMount() does not work correctly on Mac OS. The Java version does not play a role there.
        I created issue https://issues.apache.org/jira/browse/HADOOP-9337 to address the problem. In the 9337's description I suggest 2 ways of fix: can you please share your opinion on which one seems to be preferable?

        Show
        Ivan A. Veselovsky added a comment - Hi, Eric, i found the reason of the failure: o.a.h.fs.DF#getMount() does not work correctly on Mac OS. The Java version does not play a role there. I created issue https://issues.apache.org/jira/browse/HADOOP-9337 to address the problem. In the 9337's description I suggest 2 ways of fix: can you please share your opinion on which one seems to be preferable?

          People

          • Assignee:
            Ivan A. Veselovsky
            Reporter:
            Ivan A. Veselovsky
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development