XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.2, 0.10.0
    • None
    • None

    Description

      TestReduceProcessor makes an assumption about the location of pre committed output files based on fileoutputcommiter version 1. In hadoop3, the default fileoutputcommiter version moved to 2, changing the output file locations.

      org.apache.tez.mapreduce.processor.reduce.TestReduceProcessor.testReduceProcessor
      
      Failing for the past 1 build (Since Failed#2782 )
      Took 1 sec.
      Error Message
      File file:/tmp/TestReduceProcessor/output/_temporary/0/task_01_0000_r_000000/part-v001-o000-00000 does not exist
      Stacktrace
      java.io.FileNotFoundException: File file:/tmp/TestReduceProcessor/output/_temporary/0/task_01_0000_r_000000/part-v001-o000-00000 does not exist
      	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:641)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:867)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:631)
      	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
      	at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1863)
      	at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1886)
      	at org.apache.tez.mapreduce.processor.reduce.TestReduceProcessor.testReduceProcessor(TestReduceProcessor.java:259)
      

      Attachments

        1. TEZ-3927.001.patch
          1 kB
          Jonathan Turner Eagles
        2. TEZ-3927.002.patch
          1 kB
          Jonathan Turner Eagles

        Activity

          People

            jeagles Jonathan Turner Eagles
            jeagles Jonathan Turner Eagles
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: