Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3849

Too much of preemption activity causing continuos killing of containers across queues

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Two queues are used. Each queue has given a capacity of 0.5. Dominant Resource policy is used.

      1. An app is submitted in QueueA which is consuming full cluster capacity
      2. After submitting an app in QueueB, there are some demand and invoking preemption in QueueA
      3. Instead of killing the excess of 0.5 guaranteed capacity, we observed that all containers other than AM is getting killed in QueueA
      4. Now the app in QueueB is trying to take over cluster with the current free space. But there are some updated demand from the app in QueueA which lost its containers earlier, and preemption is kicked in QueueB now.

      Scenario in step 3 and 4 continuously happening in loop. Thus none of the apps are completing.

        Attachments

        1. 0001-YARN-3849.patch
          54 kB
          Sunil Govindan
        2. 0002-YARN-3849.patch
          29 kB
          Sunil Govindan
        3. 0003-YARN-3849.patch
          30 kB
          Sunil Govindan
        4. 0004-YARN-3849.patch
          30 kB
          Sunil Govindan
        5. 0004-YARN-3849-branch2-7.patch
          21 kB
          Sunil Govindan
        6. 0004-YARN-3849-branch2-6.patch
          16 kB
          Sunil Govindan

          Activity

            People

            • Assignee:
              sunilg Sunil Govindan
              Reporter:
              sunilg Sunil Govindan
            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: