Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6831

Miscellaneous refactoring changes of ContainScheduler

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:

      Description

      While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler

      *Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled.

      *ContainerScheduler
      ##Why do we need maxOppQueueLength given queuingLimit?
      ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic?
      ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here?
      ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary

      *OpportunisticContainersStatus
      ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated?
      ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive.
      ##Have we considered folding ContainerQueuingLimit class into this?

      We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work.

        Attachments

          Activity

            People

            • Assignee:
              haibochen Haibo Chen
              Reporter:
              haibochen Haibo Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: