Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3508

Prevent processing preemption events on the main RM dispatcher

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We recently saw the RM for a large cluster lag far behind on the AsyncDispacher event queue. The AsyncDispatcher thread was consistently blocked on the highly-contended CapacityScheduler lock trying to dispatch preemption-related events for RMContainerPreemptEventDispatcher. Preemption processing should occur on the scheduler event dispatcher thread or a separate thread to avoid delaying the processing of other events in the primary dispatcher queue.

        Attachments

        1. YARN-3508.002.patch
          14 kB
          Varun Saxena
        2. YARN-3508.01.patch
          13 kB
          Varun Saxena
        3. YARN-3508.03.patch
          26 kB
          Varun Saxena
        4. YARN-3508.04.patch
          26 kB
          Varun Saxena
        5. YARN-3508.05.patch
          33 kB
          Varun Saxena
        6. YARN-3508.06.patch
          33 kB
          Varun Saxena
        7. YARN-3508-branch-2.7.01.patch
          29 kB
          Varun Saxena

          Activity

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: