Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1336

[Umbrella] Work-preserving nodemanager restart

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • None
    • graceful, nodemanager, rolling upgrade
    • None

    Description

      This serves as an umbrella ticket for tasks related to work-preserving nodemanager restart.

      Attachments

        1. NMRestartDesignOverview.pdf
          146 kB
          Jason Darrell Lowe
        2. YARN-1336-rollup.patch
          237 kB
          Jason Darrell Lowe
        3. YARN-1336-rollup-v2.patch
          108 kB
          Jason Darrell Lowe

        Issue Links

          causes

          New Feature - A new feature of the product, which has yet to be developed. YARN-8012 Support Unmanaged Container Cleanup

          • Major - Major loss of function.
          • Patch Available
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. YARN-666 [Umbrella] Support rolling upgrades in YARN

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue MAPREDUCE-5891 Improved shuffle error handling across NM restarts

          • Major - Major loss of function.
          • Closed
          is required by

          Sub-task - The sub-task of the issue YARN-6704 Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue YARN-6127 Add support for work preserving NM restart when AMRMProxy is enabled

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5652 NM Recovery. ShuffleHandler should handle NM restarts

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1593 support out-of-proc AuxiliaryServices

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-7368 Yarn Work-Preserving Better Handling Failed Disk

          • Major - Major loss of function.
          • Open
          1.
          		 Recover containers upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          2.
          		 Recover localized resource cache state upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          3.
          		 Recover DeletionService state upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          4.
          		 Recover NMTokens upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          5.
          		 Recover container tokens upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          6.
          		 Recover LogAggregationService upon nodemanager restart Sub-task Resolved Jason Darrell Lowe
          7.
          		 Recover applications upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          8.
          		 Recover application ACLs upon nodemanager restart Sub-task Resolved Jason Darrell Lowe
          9.
          		 Distinguish between nodemanager shutdown for decommission vs shutdown for restart Sub-task Closed Jason Darrell Lowe
          10.
          		 NM Recovery. Auxiliary service support. Sub-task Closed Jason Darrell Lowe
          11.
          		 Recover information about finished containers Sub-task Resolved Unassigned
          12.
          		 Recover NonAggregatingLogHandler state upon nodemanager restart Sub-task Closed Jason Darrell Lowe
          13.
          		 Metrics recovery for nodemanager restart Sub-task Open Unassigned
          14.
          		 Distinguish shutdown during supervision vs. shutdown for rolling upgrade Sub-task Resolved Jason Darrell Lowe
          15.
          		 NM restart: Container recovery for Windows Sub-task In Progress Yuqi Wang
          16.
          		 NM restart: cgroup is not removed for reacquired containers Sub-task Closed Jason Darrell Lowe
          17.
          		 Document NM Restart feature Sub-task Closed Junping Du
          18.
          		 Recover appTokenKeepAliveMap upon nodemanager restart Sub-task Resolved Unassigned

          Activity

            People

              jlowe Jason Darrell Lowe
              jlowe Jason Darrell Lowe
              Votes:
              1 Vote for this issue
              Watchers:
              48 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: