Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5196

CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha1
    • mr-am, mrv2
    • None
    • Reviewed

    Description

      This JIRA tracks a checkpoint-based AM preemption policy. The policy handles propagation of the preemption requests received from the RM to the appropriate tasks, and bookeeping of checkpoints. Actual checkpointing of the task state is handled in upcoming JIRAs.

      Attachments

        1. MAPREDUCE-5196.patch
          58 kB
          Carlo Curino
        2. MAPREDUCE-5196.patch
          58 kB
          Carlo Curino
        3. MAPREDUCE-5196.1.patch
          73 kB
          Carlo Curino
        4. MAPREDUCE-5196.2.patch
          69 kB
          Christopher Douglas
        5. MAPREDUCE-5196.3.patch
          79 kB
          Christopher Douglas

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5717 Task pings are interpreted as task progress

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5912 Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196

          • Major - Major loss of function.
          • Resolved
          depends upon

          Sub-task - The sub-task of the issue MAPREDUCE-5189 Basic AM changes to support preemption requests (per YARN-45)

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5197 Checkpoint Service: a library component to facilitate checkpoint of task state

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5269 Preemption of Reducer (and Shuffle) via checkpointing

          • Major - Major loss of function.
          • Patch Available
          is part of

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-4584 Umbrella: Preemption and restart of MapReduce tasks

          • Major - Major loss of function.
          • Open
          is related to

          Sub-task - The sub-task of the issue MAPREDUCE-5176 Preemptable annotations (to support preemption in MR)

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5197 Checkpoint Service: a library component to facilitate checkpoint of task state

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              curino Carlo Curino
              curino Carlo Curino
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: