Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10515

Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The testcase TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags sometimes fails with the following error:

      org.apache.hadoop.service.ServiceStateException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to initialize queues
      	at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:174)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:110)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:884)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:165)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
      	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.serviceInit(MockRM.java:1018)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:165)
      	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:158)
      	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:134)
      	at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:130)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation$5.<init>(TestCapacitySchedulerAutoQueueCreation.java:873)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags(TestCapacitySchedulerAutoQueueCreation.java:873)
              ...
      Caused by: org.apache.hadoop.metrics2.MetricsException: Metrics source PartitionQueueMetrics,partition=,q0=root,q1=a already exists!
      	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
      	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
      	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.getPartitionQueueMetrics(QueueMetrics.java:317)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.setAvailableResourcesToQueue(QueueMetrics.java:513)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils.updateQueueStatistics(CSQueueUtils.java:308)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractCSQueue.setupQueueConfigs(AbstractCSQueue.java:412)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractCSQueue.setupQueueConfigs(AbstractCSQueue.java:350)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setupQueueConfigs(ParentQueue.java:137)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.<init>(ParentQueue.java:119)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractManagedParentQueue.<init>(AbstractManagedParentQueue.java:52)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ManagedParentQueue.<init>(ManagedParentQueue.java:57)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:261)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:289)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.initializeQueues(CapacitySchedulerQueueManager.java:162)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:752)
      

      We have to reset queue metrics before running this test to make sure it passes.

        Attachments

        1. YARN-10515-001.patch
          2 kB
          Peter Bacsko

          Activity

            People

            • Assignee:
              pbacsko Peter Bacsko
              Reporter:
              pbacsko Peter Bacsko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: