Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3508

Prevent processing preemption events on the main RM dispatcher

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We recently saw the RM for a large cluster lag far behind on the AsyncDispacher event queue. The AsyncDispatcher thread was consistently blocked on the highly-contended CapacityScheduler lock trying to dispatch preemption-related events for RMContainerPreemptEventDispatcher. Preemption processing should occur on the scheduler event dispatcher thread or a separate thread to avoid delaying the processing of other events in the primary dispatcher queue.

        Attachments

        1. YARN-3508.002.patch
          14 kB
          Varun Saxena
        2. YARN-3508.01.patch
          13 kB
          Varun Saxena
        3. YARN-3508.03.patch
          26 kB
          Varun Saxena
        4. YARN-3508.04.patch
          26 kB
          Varun Saxena
        5. YARN-3508.05.patch
          33 kB
          Varun Saxena
        6. YARN-3508.06.patch
          33 kB
          Varun Saxena
        7. YARN-3508-branch-2.7.01.patch
          29 kB
          Varun Saxena

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              jlowe Jason Darrell Lowe

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment