Uploaded image for project: 'REEF (Retired)'
  1. REEF (Retired)
  2. REEF-345 Complete implementation for YARN AM HA
  3. REEF-85

Refactor the Driver restart code to be modular

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • None
    • 0.12
    • None

    Description

      During REEF-80, we discovered that the Driver today keeps a log of all Evaluator (de-)allocations on HDFS for use during a Driver restart situation. This is happening even for Drivers that don't make use of the Driver restart feature. We should refactor this code to be modular, not to impose this cost on all Drivers.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue REEF-483 Add configuration in allowing other ways to preserve running evaluators aside from DFS

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              afchung90 Andrew Chung
              markus.weimer Markus Weimer
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: