Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5542 Scheduling of opportunistic containers
  3. YARN-9697

Efficient allocation of Opportunistic containers.

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
      None

      Description

      In the current implementation, opportunistic containers are allocated based on the number of queued opportunistic container information received in node heartbeat. This information becomes stale as soon as more opportunistic containers are allocated on that node.

      Allocation of opportunistic containers happens on the same heartbeat in which AM asks for the containers. When multiple applications request for Opportunistic containers, containers might get allocated on the same set of nodes as already allocated containers on the node are not considered while serving requests from different applications. This can lead to uneven allocation of Opportunistic containers across the cluster leading to increased queuing time 

        Attachments

        1. YARN-9697.wip2.patch
          41 kB
          Abhishek Modi
        2. YARN-9697.wip1.patch
          98 kB
          Abhishek Modi
        3. YARN-9697.ut2.patch
          4 kB
          Abhishek Modi
        4. YARN-9697.ut.patch
          4 kB
          Abhishek Modi
        5. YARN-9697.009.patch
          70 kB
          Abhishek Modi
        6. YARN-9697.008.patch
          69 kB
          Abhishek Modi
        7. YARN-9697.007.patch
          68 kB
          Abhishek Modi
        8. YARN-9697.006.patch
          68 kB
          Abhishek Modi
        9. YARN-9697.005.patch
          68 kB
          Abhishek Modi
        10. YARN-9697.004.patch
          65 kB
          Abhishek Modi
        11. YARN-9697.003.patch
          82 kB
          Abhishek Modi
        12. YARN-9697.002.patch
          82 kB
          Abhishek Modi
        13. YARN-9697.001.patch
          81 kB
          Abhishek Modi

          Issue Links

            Activity

              People

              • Assignee:
                abmodi Abhishek Modi
                Reporter:
                abmodi Abhishek Modi
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: