Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-914 (Umbrella) Support graceful decommission of nodemanager
  3. YARN-9608

DecommissioningNodesWatcher should get lists of running applications on node from RMNode.

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
      None

      Description

      At present, DecommissioningNodesWatcher tracks list of running applications and triggers decommission of nodes when all the applications that ran on the node completes. This Jira proposes to solve following problem:

      1. DecommissioningNodesWatcher skips tracking application containers on a particular node before the node is in DECOMMISSIONING state. It only tracks containers once the node is in DECOMMISSIONING state. This can lead to shuffle data loss of apps whose containers ran on this node before it was moved to decommissioning state.
      2. It is keeping track of running apps. We can leverage this directly from RMNode.

        Attachments

        1. YARN-9608.001.patch
          14 kB
          Abhishek Modi
        2. YARN-9608.002.patch
          13 kB
          Abhishek Modi
        3. YARN-9608-branch-3.001.patch
          14 kB
          Abhishek Modi

          Activity

            People

            • Assignee:
              abmodi Abhishek Modi
              Reporter:
              abmodi Abhishek Modi
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: