Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10447

TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.4.0
    • test
    • None
    • Reviewed

    Description

      YARN-9784 fixed some concurrency related issues in TestLeafQueue, but not all of them. Occasionally it's still possible to receive an exception from Mockito and the two following stack traces can be observed in the console:

      	
      	org.mockito.exceptions.misusing.WrongTypeOfReturnValue: 
      Integer cannot be returned by isMultiNodePlacementEnabled()
      isMultiNodePlacementEnabled() should return boolean
      ***
      If you're unsure why you're getting above error read on.
      Due to the nature of the syntax above problem might occur because:
      1. This exception *might* occur in wrongly written multi-threaded tests.
         Please refer to Mockito FAQ on limitations of concurrency testing.
      2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub spies - 
         - with doReturn|Throw() family of methods. More in javadocs for Mockito.spy() method.
      

      or

      2020-09-22 14:44:52,584 INFO  [main] capacity.TestUtils (TestUtils.java:getMockNode(227)) - node = 127.0.0.3 avail=<memory:8192, vCores:1>
      2020-09-22 14:44:52,585 INFO  [main] capacity.TestUtils (TestUtils.java:getMockNode(227)) - node = 127.0.0.4 avail=<memory:8192, vCores:1>
      Exception in thread "ActivitiesManager thread." java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Boolean
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$$EnhancerByMockitoWithCGLIB$$272c72c5.isMultiNodePlacementEnabled(<generated>)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.dynamicallyUpdateAppActivitiesMaxQueueLengthIfNeeded(ActivitiesManager.java:266)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.access$500(ActivitiesManager.java:63)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager$1.run(ActivitiesManager.java:347)
      	at java.lang.Thread.run(Thread.java:748)
      

      It's probably best to disable ActivitiesManager thread entirely in this test class, there is no need for it.

      Attachments

        1. YARN-10447-001.patch
          4 kB
          Peter Bacsko
        2. YARN-10447-002.patch
          4 kB
          Peter Bacsko
        3. YARN-10447-003.patch
          4 kB
          Peter Bacsko

        Issue Links

          Activity

            People

              pbacsko Peter Bacsko
              pbacsko Peter Bacsko
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: