Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10589

Improve logic of multi-node allocation

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      for (String partititon : partitions) {
       if (current++ > start) {
       break;
       }
       CandidateNodeSet<FiCaSchedulerNode> candidates =
       cs.getCandidateNodeSet(partititon);
       if (candidates == null) {
       continue;
       }
       cs.allocateContainersToNode(candidates, false);
      }

      In above logic, if we have thousands of node in one partition, we will still repeatedly access all nodes of the partition thousands of times. There is no break point where if the partition is not same for the first node, it should stop checking other nodes in that partition.

        Attachments

        1. YARN-10589-005.patch
          13 kB
          Tanu Ajmera
        2. YARN-10589-004.patch
          13 kB
          Tanu Ajmera
        3. YARN-10589-003.patch
          5 kB
          Qi Zhu
        4. YARN-10589-002.patch
          5 kB
          Qi Zhu
        5. YARN-10589-001.patch
          11 kB
          Tanu Ajmera

          Activity

            People

            • Assignee:
              tanu.ajmera Tanu Ajmera
              Reporter:
              tanu.ajmera Tanu Ajmera
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: