Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10501

Can't remove all node labels after add node label without nodemanager port

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.4.0
    • Fix Version/s: 3.4.0, 3.3.1, 3.1.5, 2.10.2, 3.2.3
    • Component/s: yarn
    • Labels:
      None

      Description

      When add a label to nodes without nodemanager port or use WILDCARD_PORT (0) port, it can't remove all label info in these nodes

      Reproduce process:

      1.yarn rmadmin -addToClusterNodeLabels "cpunode(exclusive=true)"
      2.yarn rmadmin -replaceLabelsOnNode "server001=cpunode"
      3.curl http://RM_IP:8088/ws/v1/cluster/label-mappings
      {"labelsToNodes":{"entry":{"key":{"name":"cpunode","exclusivity":"true"},"value":{"nodes":["server001:0","server001:45454"],"partitionInfo":{"resourceAvailable":{"memory":"510","vCores":"1","resourceInformations":{"resourceInformation":[{"attributes":null,"maximumAllocation":"9223372036854775807","minimumAllocation":"0","name":"memory-mb","resourceType":"COUNTABLE","units":"Mi","value":"510"},{"attributes":null,"maximumAllocation":"9223372036854775807","minimumAllocation":"0","name":"vcores","resourceType":"COUNTABLE","units":"","value":"1"}]}}}}}}}
      
      4.yarn rmadmin -replaceLabelsOnNode "server001"
      5.curl http://RM_IP:8088/ws/v1/cluster/label-mappings
      {"labelsToNodes":{"entry":{"key":{"name":"cpunode","exclusivity":"true"},"value":{"nodes":"server001:45454","partitionInfo":{"resourceAvailable":{"memory":"0","vCores":"0","resourceInformations":{"resourceInformation":[{"attributes":null,"maximumAllocation":"9223372036854775807","minimumAllocation":"0","name":"memory-mb","resourceType":"COUNTABLE","units":"Mi","value":"0"},{"attributes":null,"maximumAllocation":"9223372036854775807","minimumAllocation":"0","name":"vcores","resourceType":"COUNTABLE","units":"","value":"0"}]}}}}}}} 

      You can see after the 4 process to remove nodemanager labels, the label info is still in the node info.

       641 case REPLACE:
       642 replaceNodeForLabels(nodeId, host.labels, labels);
       643 replaceLabelsForNode(nodeId, host.labels, labels);
       644 host.labels.clear();
       645 host.labels.addAll(labels);
       646 for (Node node : host.nms.values()) {
       647 replaceNodeForLabels(node.nodeId, node.labels, labels);
       649 node.labels = null;
       650 }
       651 break;

      The cause is in 647 line, when add labels to node without port, the 0 port and the real nm port with be both add to node info, and when remove labels, the parameter node.labels in 647 line is null, so it will not remove the old label. 

        Attachments

        1. YARN-10502-branch-2.10.003.patch
          5 kB
          caozhiqiang
        2. YARN-10502-branch-2.10.002.patch
          5 kB
          caozhiqiang
        3. YARN-10501-branch-2.10.001.patch
          5 kB
          caozhiqiang
        4. YARN-10501.004.patch
          5 kB
          caozhiqiang
        5. YARN-10501.003.patch
          4 kB
          caozhiqiang
        6. YARN-10501.002.patch
          3 kB
          caozhiqiang

          Issue Links

            Activity

              People

              • Assignee:
                caozhiqiang caozhiqiang
                Reporter:
                caozhiqiang caozhiqiang
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: