Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5542 Scheduling of opportunistic containers
  3. YARN-9697

Efficient allocation of Opportunistic containers.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0
    • None
    • None

    Description

      In the current implementation, opportunistic containers are allocated based on the number of queued opportunistic container information received in node heartbeat. This information becomes stale as soon as more opportunistic containers are allocated on that node.

      Allocation of opportunistic containers happens on the same heartbeat in which AM asks for the containers. When multiple applications request for Opportunistic containers, containers might get allocated on the same set of nodes as already allocated containers on the node are not considered while serving requests from different applications. This can lead to uneven allocation of Opportunistic containers across the cluster leading to increased queuing time 

      Attachments

        1. YARN-9697.ut.patch
          4 kB
          Abhishek Modi
        2. YARN-9697.ut2.patch
          4 kB
          Abhishek Modi
        3. YARN-9697.wip1.patch
          98 kB
          Abhishek Modi
        4. YARN-9697.wip2.patch
          41 kB
          Abhishek Modi
        5. YARN-9697.001.patch
          81 kB
          Abhishek Modi
        6. YARN-9697.002.patch
          82 kB
          Abhishek Modi
        7. YARN-9697.003.patch
          82 kB
          Abhishek Modi
        8. YARN-9697.004.patch
          65 kB
          Abhishek Modi
        9. YARN-9697.005.patch
          68 kB
          Abhishek Modi
        10. YARN-9697.006.patch
          68 kB
          Abhishek Modi
        11. YARN-9697.007.patch
          68 kB
          Abhishek Modi
        12. YARN-9697.008.patch
          69 kB
          Abhishek Modi
        13. YARN-9697.009.patch
          70 kB
          Abhishek Modi

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. YARN-10040 DistributedShell test failure on X86 and ARM

          • Major - Major loss of function.
          • Resolved
          depends upon

          Sub-task - The sub-task of the issue YARN-9859 Refactor OpportunisticContainerAllocator

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              abmodi Abhishek Modi
              abmodi Abhishek Modi
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: