Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6009

Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slot

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.2.1
    • Fix Version/s: 1.3.0, 1.2.2
    • Component/s: client, job submission
    • Labels:
      None

      Description

      In branch 1 job commit is executed in a JOB_CLEANUP task that may run in either map or reduce slot

      in org.apache.hadoop.mapreduce.Job#setUseNewAPI there is a logic setting new-api flag only for reduce-ful jobs.

          if (numReduces != 0) {
            conf.setBooleanIfUnset("mapred.reducer.new-api",
                                   conf.get(oldReduceClass) == null);
            ...
      

      Therefore, when cleanup runs in a reduce slot, ReduceTask inits using the old API and runs incorrect default OutputCommitter, instead of consulting OutputFormat.

        Activity

        Hide
        Gera Shegalov added a comment -

        A workaround is to manually add mapred.reducer.new-api=true to the job conf.

        Show
        Gera Shegalov added a comment - A workaround is to manually add mapred.reducer.new-api=true to the job conf.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12658096/MAPREDUCE-6009.v01-branch-1.2.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4773//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12658096/MAPREDUCE-6009.v01-branch-1.2.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4773//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Is this an issue in Hadoop 2 as well?

        Show
        Sandy Ryza added a comment - Is this an issue in Hadoop 2 as well?
        Hide
        Gera Shegalov added a comment -

        Hi Sandy Ryza, in Hadoop 2, job commit is executed in AM. Hadoop 2 is not affected.

        Show
        Gera Shegalov added a comment - Hi Sandy Ryza , in Hadoop 2, job commit is executed in AM. Hadoop 2 is not affected.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12658096/MAPREDUCE-6009.v01-branch-1.2.patch
        against trunk revision 596702a.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4950//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12658096/MAPREDUCE-6009.v01-branch-1.2.patch against trunk revision 596702a. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4950//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Just to be conservative with changes, could we leave these in the if statement?

        +    if (conf.getUseNewReducer()) {
        +      String mode = "new reduce API";
        +      ensureNotSet("mapred.output.format.class", mode);
        +      ensureNotSet(oldReduceClass, mode);
        +    } else {
        +      String mode = "reduce compatability";
        +      ensureNotSet(JobContext.OUTPUT_FORMAT_CLASS_ATTR, mode);
        +      ensureNotSet(JobContext.REDUCE_CLASS_ATTR, mode);
        +    }
        
        Show
        Sandy Ryza added a comment - Just to be conservative with changes, could we leave these in the if statement? + if (conf.getUseNewReducer()) { + String mode = " new reduce API" ; + ensureNotSet( "mapred.output.format.class" , mode); + ensureNotSet(oldReduceClass, mode); + } else { + String mode = "reduce compatability" ; + ensureNotSet(JobContext.OUTPUT_FORMAT_CLASS_ATTR, mode); + ensureNotSet(JobContext.REDUCE_CLASS_ATTR, mode); + }
        Hide
        Gera Shegalov added a comment -

        Yes, this works and corresponds to our workaround. Updating the patch, accordingly.

        Show
        Gera Shegalov added a comment - Yes, this works and corresponds to our workaround. Updating the patch, accordingly.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674127/MAPREDUCE-6009.v02-branch-1.2.patch
        against trunk revision cb81bac.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4951//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674127/MAPREDUCE-6009.v02-branch-1.2.patch against trunk revision cb81bac. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4951//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Thanks, this looks good to me. Have you been able to run tests?

        Show
        Sandy Ryza added a comment - Thanks, this looks good to me. Have you been able to run tests?
        Hide
        Gera Shegalov added a comment -

        I ran tests, and got two unrelated errors/failures.

        $ ant test
        $ find . -name TEST\*txt | xargs grep 'Tests run' | grep 'Failures: [^0]\|Errors: [^0]' 
        ./build/test/TEST-org.apache.hadoop.mapred.TestJobRetire.txt:Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
        ./build/test/TEST-org.apache.hadoop.tools.rumen.TestRumenJobTraces.txt:Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 27.265 sec
        
        Show
        Gera Shegalov added a comment - I ran tests, and got two unrelated errors/failures. $ ant test $ find . -name TEST\*txt | xargs grep 'Tests run' | grep 'Failures: [^0]\|Errors: [^0]' ./build/test/TEST-org.apache.hadoop.mapred.TestJobRetire.txt:Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec ./build/test/TEST-org.apache.hadoop.tools.rumen.TestRumenJobTraces.txt:Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 27.265 sec
        Hide
        Sandy Ryza added a comment -

        +1

        Show
        Sandy Ryza added a comment - +1

          People

          • Assignee:
            Gera Shegalov
            Reporter:
            Gera Shegalov
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development