Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: applicationmaster, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Enhance OutputCommitter and FileOutputCommitter to allow for recover of tasks across job restart.

      Description

      In MR AM recovers from a crash, it only reruns the non completed tasks. The completed tasks (along with their output, if any) needs to be recovered from the previous life. This would require some changes in OutputCommitter.

      1. mr2702_v1.patch
        26 kB
        Sharad Agarwal
      2. mr2702_v2.patch
        20 kB
        Sharad Agarwal
      3. mr2702_v3.patch
        20 kB
        Sharad Agarwal
      4. mr2702_v4.patch
        20 kB
        Sharad Agarwal
      5. MAPREDUCE-2702.patch
        20 kB
        Arun C Murthy
      6. MAPREDUCE-2702.patch
        21 kB
        Arun C Murthy
      7. MAPREDUCE-2702.patch
        21 kB
        Arun C Murthy

        Issue Links

          Activity

          Sharad Agarwal created issue -
          Sharad Agarwal made changes -
          Field Original Value New Value
          Link This issue blocks MAPREDUCE-2708 [ MAPREDUCE-2708 ]
          Sharad Agarwal made changes -
          Summary OutputCommitter changes for MR Application Master recovery [MR-279] OutputCommitter changes for MR Application Master recovery
          Sharad Agarwal made changes -
          Attachment mr2702_v1.patch [ 12489459 ]
          Sharad Agarwal made changes -
          Attachment mr2702_v2.patch [ 12491326 ]
          Sharad Agarwal made changes -
          Attachment mr2702_v3.patch [ 12491601 ]
          Sharad Agarwal made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Sharad Agarwal made changes -
          Attachment mr2702_v4.patch [ 12494030 ]
          Arun C Murthy made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Arun C Murthy made changes -
          Attachment MAPREDUCE-2702.patch [ 12497598 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Arun C Murthy made changes -
          Attachment MAPREDUCE-2702.patch [ 12497751 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Arun C Murthy made changes -
          Attachment MAPREDUCE-2702.patch [ 12497765 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.23.0 [ 12315570 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Affects Version/s 0.23.0 [ 12315570 ]
          Component/s applicationmaster [ 12315625 ]
          Vinod Kumar Vavilapalli made changes -
          Parent MAPREDUCE-2692 [ 12514289 ]
          Issue Type New Feature [ 2 ] Sub-task [ 7 ]
          Arun C Murthy made changes -
          Release Note Enhance OutputCommitter and FileOutputCommitter to allow for recover of tasks across job restart.
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Sharad Agarwal
              Reporter:
              Sharad Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development