Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-947

OutputCommitter should have an abortJob method

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      Introduced abortJob() method in OutputCommitter which will be invoked when the job fails or is killed. By default it invokes OutputCommitter.cleanupJob(). Deprecated OutputCommitter.cleanupJob() and introduced OutputCommitter.commitJob() method which will be invoked for successful jobs. Also a _SUCCESS file is created in the output folder for successful jobs. A configuration parameter mapreduce.fileoutputcommitter.marksuccessfuljobs can be set to false to disable creation of _SUCCESS file, or to true to enable creation of the _SUCCESS file.
      Show
      Introduced abortJob() method in OutputCommitter which will be invoked when the job fails or is killed. By default it invokes OutputCommitter.cleanupJob(). Deprecated OutputCommitter.cleanupJob() and introduced OutputCommitter.commitJob() method which will be invoked for successful jobs. Also a _SUCCESS file is created in the output folder for successful jobs. A configuration parameter mapreduce.fileoutputcommitter.marksuccessfuljobs can be set to false to disable creation of _SUCCESS file, or to true to enable creation of the _SUCCESS file.

      Description

      The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently there is no way to distinguish between failed or succeeded jobs, making it impossible to write output promotion code.

      1. yhadoop20-bug-fix-947.patch
        0.6 kB
        Ravi Gummadi
      2. mr-947-y20.patch
        69 kB
        Jothi Padmanabhan
      3. mr-947-trunk.patch
        84 kB
        Arun C Murthy
      4. mr-947-y20-new.patch
        65 kB
        Jothi Padmanabhan
      5. mr-947-trunk-new.patch
        78 kB
        Jothi Padmanabhan
      6. mr-947-y20.patch
        65 kB
        Jothi Padmanabhan
      7. mr-947-trunk-new.patch
        78 kB
        Jothi Padmanabhan
      8. mr-947-trunk.patch
        78 kB
        Jothi Padmanabhan
      9. mr-947-trunk.patch
        78 kB
        Jothi Padmanabhan
      10. mapred-948-v3.4.patch
        77 kB
        Amar Kamat
      11. mapred-948-v3.2.patch
        26 kB
        Amar Kamat
      12. mapred-948-v3.1.patch
        26 kB
        Amar Kamat
      13. mapred-948-v2.3.patch
        23 kB
        Amar Kamat
      14. mapred-948-v2.3-branch-0.20.patch
        23 kB
        Amar Kamat
      15. mapred-948-v1.13-branch-0.20-internal.patch
        23 kB
        Amar Kamat
      16. mapred-948-v2.1-branch-0.20.patch
        26 kB
        Amar Kamat
      17. mapred-948-v1.12-branch-0.20-internal.patch
        31 kB
        Amar Kamat
      18. mapred-948-v1.12.patch
        52 kB
        Amar Kamat
      19. mapred-948-v1.7.patch
        51 kB
        Amar Kamat
      20. mapred-948-v1.4.patch
        49 kB
        Amar Kamat
      21. mapred-948-v1.3.patch
        45 kB
        Amar Kamat
      22. mapred-948-v1.2.patch
        20 kB
        Amar Kamat

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development