Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7146

Many RM unit tests failing with FairScheduler

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta1
    • 2.9.0, 3.0.0-beta1, 3.1.0
    • test
    • None

    Description

      Many of the RM unit tests are failing when using the FairScheduler.

      Here is a list of affected test classes:

      TestYarnClient
      TestApplicationCleanup
      TestApplicationMasterLauncher
      TestDecommissioningNodesWatcher
      TestKillApplicationWithRMHA
      TestNodeBlacklistingOnAMFailures
      TestRM
      TestRMAdminService
      TestRMRestart
      TestResourceTrackerService
      TestWorkPreservingRMRestart
      TestAMRMRPCNodeUpdates
      TestAMRMRPCResponseId
      TestAMRestart
      TestApplicationLifetimeMonitor
      TestNodesListManager
      TestRMContainerImpl
      TestAbstractYarnScheduler
      TestSchedulerUtils
      TestFairOrderingPolicy
      TestAMRMTokens
      TestDelegationTokenRenewer
      

      Most of the test methods in these classes are failing, though some do succeed.

      There's two main categories of issues:

      1. The test submits an application to the MockRM and waits for it to enter a specific state, which it never does, and the test times out. We need to call update() on the scheduler.
      2. The test throws a ClassCastException on FSQueueMetrics to CSQueueMetrics. This is because QueueMetrics metrics are static, and a previous test using FairScheduler initialized it, and the current test is using CapacityScheduler. We need to reset the metrics.

      Attachments

        1. YARN-7146.004.patch
          52 kB
          Robert Kanter
        2. YARN-7146.004.branch-2.patch
          53 kB
          Robert Kanter
        3. YARN-7146.003.patch
          52 kB
          Robert Kanter
        4. YARN-7146.002.patch
          42 kB
          Robert Kanter
        5. YARN-7146.001.patch
          16 kB
          Robert Kanter

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rkanter Robert Kanter
            rkanter Robert Kanter
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment