Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7146

Many RM unit tests failing with FairScheduler

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-beta1
    • Fix Version/s: 2.9.0, 3.0.0-beta1, 3.1.0
    • Component/s: test
    • Labels:
      None

      Description

      Many of the RM unit tests are failing when using the FairScheduler.

      Here is a list of affected test classes:

      TestYarnClient
      TestApplicationCleanup
      TestApplicationMasterLauncher
      TestDecommissioningNodesWatcher
      TestKillApplicationWithRMHA
      TestNodeBlacklistingOnAMFailures
      TestRM
      TestRMAdminService
      TestRMRestart
      TestResourceTrackerService
      TestWorkPreservingRMRestart
      TestAMRMRPCNodeUpdates
      TestAMRMRPCResponseId
      TestAMRestart
      TestApplicationLifetimeMonitor
      TestNodesListManager
      TestRMContainerImpl
      TestAbstractYarnScheduler
      TestSchedulerUtils
      TestFairOrderingPolicy
      TestAMRMTokens
      TestDelegationTokenRenewer
      

      Most of the test methods in these classes are failing, though some do succeed.

      There's two main categories of issues:

      1. The test submits an application to the MockRM and waits for it to enter a specific state, which it never does, and the test times out. We need to call update() on the scheduler.
      2. The test throws a ClassCastException on FSQueueMetrics to CSQueueMetrics. This is because QueueMetrics metrics are static, and a previous test using FairScheduler initialized it, and the current test is using CapacityScheduler. We need to reset the metrics.

        Attachments

        1. YARN-7146.001.patch
          16 kB
          Robert Kanter
        2. YARN-7146.002.patch
          42 kB
          Robert Kanter
        3. YARN-7146.003.patch
          52 kB
          Robert Kanter
        4. YARN-7146.004.branch-2.patch
          53 kB
          Robert Kanter
        5. YARN-7146.004.patch
          52 kB
          Robert Kanter

          Activity

            People

            • Assignee:
              rkanter Robert Kanter
              Reporter:
              rkanter Robert Kanter
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: