Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestMiniMRLocalFS fails on trunk. I checked with both trunk revs - pre and post MAPREDUCE-1430 commit and it failed in both the cases.

      1. patch-1520.txt
        1 kB
        Amareshwari Sriramadasu
      2. patch-1520-1.txt
        1 kB
        Amareshwari Sriramadasu
      3. patch-1520-20S.txt
        1 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Devaraj Das created issue -
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I ran it once, here's the failure trace..

          Testcase: testWithLocal took 64.078 sec
                  Caused an ERROR
          File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist.
          java.io.FileNotFoundException: File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist.
                  at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420)
                  at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246)
                  at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1466)
                  at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1447)
                  at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:313)
                  at org.apache.hadoop.mapred.TestMiniMRWithDFS.runPI(TestMiniMRWithDFS.java:235)
                  at org.apache.hadoop.mapred.TestMiniMRLocalFS.testWithLocal(TestMiniMRLocalFS.java:68)
          

          Still wondering how Hudson didn't fail this test while blessing MAPREDUCE-1307.

          Show
          Vinod Kumar Vavilapalli added a comment - I ran it once, here's the failure trace.. Testcase: testWithLocal took 64.078 sec Caused an ERROR File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist. java.io.FileNotFoundException: File QuasiMonteCarlo_TMP_3_141592654/out/reduce-out does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:246) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1466) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1447) at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:313) at org.apache.hadoop.mapred.TestMiniMRWithDFS.runPI(TestMiniMRWithDFS.java:235) at org.apache.hadoop.mapred.TestMiniMRLocalFS.testWithLocal(TestMiniMRLocalFS.java:68) Still wondering how Hudson didn't fail this test while blessing MAPREDUCE-1307 .
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failure is triggered after the commit of HADOOP-6545. After HADOOP-6545, UGI becomes FileSystem's cache key. Changed Child.java to set working directory of the FileSystem as childUGI.

          Show
          Amareshwari Sriramadasu added a comment - Test failure is triggered after the commit of HADOOP-6545 . After HADOOP-6545 , UGI becomes FileSystem's cache key. Changed Child.java to set working directory of the FileSystem as childUGI.
          Amareshwari Sriramadasu made changes -
          Field Original Value New Value
          Attachment patch-1520.txt [ 12436674 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Link This issue relates to HADOOP-6545 [ HADOOP-6545 ]
          Hide
          Devaraj Das added a comment -

          Good catch, Amareshwari!

          Show
          Devaraj Das added a comment - Good catch, Amareshwari!
          Hide
          Amareshwari Sriramadasu added a comment -

          Moved the comment for setWorkingDirectory also; no other change.

          Show
          Amareshwari Sriramadasu added a comment - Moved the comment for setWorkingDirectory also; no other change.
          Amareshwari Sriramadasu made changes -
          Attachment patch-1520-1.txt [ 12436679 ]
          Hide
          Devaraj Das added a comment -

          +1

          Show
          Devaraj Das added a comment - +1
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436674/patch-1520.txt
          against trunk revision 912471.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436674/patch-1520.txt against trunk revision 912471. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/475/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failures TestStreamingBadRecords and TestJobHistory are not related to the patch.
          TestStreamingBadRecords failed with a ZipException. Stacktrace for the error:

          java.io.IOException: java.util.zip.ZipException: ZIP_Read: error reading zip file
          	at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:166)
          	at org.apache.hadoop.streaming.TestStreamingBadRecords.testNarrowDown(TestStreamingBadRecords.java:226)
          

          TestJobHistory failed because of HADOOP-6528. Stacktrace for the failure:

          java.lang.IllegalArgumentException: port out of range:-1
          	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:377)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.activate(NameNode.java:317)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:308)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:410)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1263)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:127)
          	at org.apache.hadoop.mapred.TestJobHistory.testDoneFolderOnHDFS(TestJobHistory.java:620)
          
          Show
          Amareshwari Sriramadasu added a comment - Test failures TestStreamingBadRecords and TestJobHistory are not related to the patch. TestStreamingBadRecords failed with a ZipException. Stacktrace for the error: java.io.IOException: java.util.zip.ZipException: ZIP_Read: error reading zip file at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:166) at org.apache.hadoop.streaming.TestStreamingBadRecords.testNarrowDown(TestStreamingBadRecords.java:226) TestJobHistory failed because of HADOOP-6528 . Stacktrace for the failure: java.lang.IllegalArgumentException: port out of range:-1 at java.net.InetSocketAddress.<init>(InetSocketAddress.java:118) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:377) at org.apache.hadoop.hdfs.server.namenode.NameNode.activate(NameNode.java:317) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:308) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:410) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1263) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:127) at org.apache.hadoop.mapred.TestJobHistory.testDoneFolderOnHDFS(TestJobHistory.java:620)
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for Yahoo! distribution

          Show
          Amareshwari Sriramadasu added a comment - Patch for Yahoo! distribution
          Amareshwari Sriramadasu made changes -
          Attachment patch-1520-20S.txt [ 12436695 ]
          Hide
          Devaraj Das added a comment -

          I don't think this requires any testcase. +1 for the patch.

          Show
          Devaraj Das added a comment - I don't think this requires any testcase. +1 for the patch.
          Hide
          Chris Douglas added a comment -

          +1 Great catch

          I committed this. Thanks, Amareshwari!

          Show
          Chris Douglas added a comment - +1 Great catch I committed this. Thanks, Amareshwari!
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Assignee Amareshwari Sriramadasu [ amareshwari ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #259 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/259/)
          . Fix TestMiniMRLocalFS failure caused by regression in
          getting user working dir. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #259 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/259/ ) . Fix TestMiniMRLocalFS failure caused by regression in getting user working dir. Contributed by Amareshwari Sriramadasu
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/)
          . Fix TestMiniMRLocalFS failure caused by regression in
          getting user working dir. Contributed by Amareshwari Sriramadasu

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/ ) . Fix TestMiniMRLocalFS failure caused by regression in getting user working dir. Contributed by Amareshwari Sriramadasu
          Tom White made changes -
          Fix Version/s 0.22.0 [ 12314184 ]
          Fix Version/s 0.21.0 [ 12314045 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          22h 44m 1 Amareshwari Sriramadasu 23/Feb/10 04:56
          Patch Available Patch Available Resolved Resolved
          9d 1h 20m 1 Chris Douglas 04/Mar/10 06:17
          Resolved Resolved Closed Closed
          173d 15h 3m 1 Tom White 24/Aug/10 22:20

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development