Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2692 Ensure AM Restart and Recovery-on-restart is complete
  3. MAPREDUCE-2702

[MR-279] OutputCommitter changes for MR Application Master recovery

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.0
    • 0.23.0
    • applicationmaster, mrv2
    • None
    • Reviewed
    • Enhance OutputCommitter and FileOutputCommitter to allow for recover of tasks across job restart.

    Description

      In MR AM recovers from a crash, it only reruns the non completed tasks. The completed tasks (along with their output, if any) needs to be recovered from the previous life. This would require some changes in OutputCommitter.

      Attachments

        1. mr2702_v1.patch
          26 kB
          Sharad Agarwal
        2. mr2702_v2.patch
          20 kB
          Sharad Agarwal
        3. mr2702_v3.patch
          20 kB
          Sharad Agarwal
        4. mr2702_v4.patch
          20 kB
          Sharad Agarwal
        5. MAPREDUCE-2702.patch
          20 kB
          Arun Murthy
        6. MAPREDUCE-2702.patch
          21 kB
          Arun Murthy
        7. MAPREDUCE-2702.patch
          21 kB
          Arun Murthy

        Issue Links

          blocks

          Sub-task - The sub-task of the issue MAPREDUCE-2708 [MR-279] Design and implement MR Application Master recovery

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              sharadag Sharad Agarwal
              sharadag Sharad Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: