Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestMiniMRLocalFS fails on trunk. I checked with both trunk revs - pre and post MAPREDUCE-1430 commit and it failed in both the cases.

      1. patch-1520-20S.txt
        1 kB
        Amareshwari Sriramadasu
      2. patch-1520-1.txt
        1 kB
        Amareshwari Sriramadasu
      3. patch-1520.txt
        1 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Vinod Kumar Vavilapalli added a comment -

          I ran it once, here's the failure trace..

          Testcase: testWithLocal took 64.078 sec
                  Caused an ERROR
          File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist.
          java.io.FileNotFoundException: File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist.
                  at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420)
                  at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246)
                  at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1466)
                  at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1447)
                  at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:313)
                  at org.apache.hadoop.mapred.TestMiniMRWithDFS.runPI(TestMiniMRWithDFS.java:235)
                  at org.apache.hadoop.mapred.TestMiniMRLocalFS.testWithLocal(TestMiniMRLocalFS.java:68)
          

          Still wondering how Hudson didn't fail this test while blessing MAPREDUCE-1307.

          Show
          Vinod Kumar Vavilapalli added a comment - I ran it once, here's the failure trace.. Testcase: testWithLocal took 64.078 sec Caused an ERROR File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist. java.io.FileNotFoundException: File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1466) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1447) at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:313) at org.apache.hadoop.mapred.TestMiniMRWithDFS.runPI(TestMiniMRWithDFS.java:235) at org.apache.hadoop.mapred.TestMiniMRLocalFS.testWithLocal(TestMiniMRLocalFS.java:68) Still wondering how Hudson didn't fail this test while blessing MAPREDUCE-1307 .
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failure is triggered after the commit of HADOOP-6545. After HADOOP-6545, UGI becomes FileSystem's cache key. Changed Child.java to set working directory of the FileSystem as childUGI.

          Show
          Amareshwari Sriramadasu added a comment - Test failure is triggered after the commit of HADOOP-6545 . After HADOOP-6545 , UGI becomes FileSystem's cache key. Changed Child.java to set working directory of the FileSystem as childUGI.
          Hide
          Devaraj Das added a comment -

          Good catch, Amareshwari!

          Show
          Devaraj Das added a comment - Good catch, Amareshwari!
          Hide
          Amareshwari Sriramadasu added a comment -

          Moved the comment for setWorkingDirectory also; no other change.

          Show
          Amareshwari Sriramadasu added a comment - Moved the comment for setWorkingDirectory also; no other change.
          Hide
          Devaraj Das added a comment -

          +1

          Show
          Devaraj Das added a comment - +1
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436674/patch-1520.txt
          against trunk revision 912471.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436674/patch-1520.txt against trunk revision 912471. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failures TestStreamingBadRecords and TestJobHistory are not related to the patch.
          TestStreamingBadRecords failed with a ZipException. Stacktrace for the error:

          java.io.IOException: java.util.zip.ZipException: ZIP_Read: error reading zip file
          	at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:166)
          	at org.apache.hadoop.streaming.TestStreamingBadRecords.testNarrowDown(TestStreamingBadRecords.java:226)
          

          TestJobHistory failed because of HADOOP-6528. Stacktrace for the failure:

          java.lang.IllegalArgumentException: port out of range:-1
          	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:377)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.activate(NameNode.java:317)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:308)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:410)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1263)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:127)
          	at org.apache.hadoop.mapred.TestJobHistory.testDoneFolderOnHDFS(TestJobHistory.java:620)
          
          Show
          Amareshwari Sriramadasu added a comment - Test failures TestStreamingBadRecords and TestJobHistory are not related to the patch. TestStreamingBadRecords failed with a ZipException. Stacktrace for the error: java.io.IOException: java.util.zip.ZipException: ZIP_Read: error reading zip file at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:166) at org.apache.hadoop.streaming.TestStreamingBadRecords.testNarrowDown(TestStreamingBadRecords.java:226) TestJobHistory failed because of HADOOP-6528 . Stacktrace for the failure: java.lang.IllegalArgumentException: port out of range:-1 at java.net.InetSocketAddress.<init>(InetSocketAddress.java:118) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:377) at org.apache.hadoop.hdfs.server.namenode.NameNode.activate(NameNode.java:317) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:308) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:410) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1263) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:127) at org.apache.hadoop.mapred.TestJobHistory.testDoneFolderOnHDFS(TestJobHistory.java:620)
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for Yahoo! distribution

          Show
          Amareshwari Sriramadasu added a comment - Patch for Yahoo! distribution
          Hide
          Devaraj Das added a comment -

          I don't think this requires any testcase. +1 for the patch.

          Show
          Devaraj Das added a comment - I don't think this requires any testcase. +1 for the patch.
          Hide
          Chris Douglas added a comment -

          +1 Great catch

          I committed this. Thanks, Amareshwari!

          Show
          Chris Douglas added a comment - +1 Great catch I committed this. Thanks, Amareshwari!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #259 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/259/)
          . Fix TestMiniMRLocalFS failure caused by regression in
          getting user working dir. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #259 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/259/ ) . Fix TestMiniMRLocalFS failure caused by regression in getting user working dir. Contributed by Amareshwari Sriramadasu
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/)
          . Fix TestMiniMRLocalFS failure caused by regression in
          getting user working dir. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/ ) . Fix TestMiniMRLocalFS failure caused by regression in getting user working dir. Contributed by Amareshwari Sriramadasu

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development