Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-2111

ManagementContext can leak

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.6.3
    • 2.6.4, 2.7.0
    • None
    • None

    Description

      If a management-context connector is configured in management.xml (e.g. below) and org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl#stop(boolean, boolean, boolean) is called (e.g. due to criticalIOErrors) the JVM will not exit due to a handful of leaked RMI related threads for the MBean server implementation.

      <management-context xmlns="http://activemq.org/schema">
         <connector connector-port="1099" connector-host="0.0.0.0"/>
         ...
      </management-context>
      

      Here's an example thread dump after the broker has stopped:

      2018-08-10 10:47:59
      Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.20-b23 mixed mode):
      
      "Attach Listener" #122 daemon prio=9 os_prio=31 tid=0x00007fe609a37800 nid=0x920f waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "DestroyJavaVM" #57 prio=5 os_prio=31 tid=0x00007fe60a000800 nid=0x1c03 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "RMI RenewClean-[192.168.1.125:62416]" #20 daemon prio=5 os_prio=31 tid=0x00007fe609432800 nid=0x7f03 in Object.wait() [0x000070000672e000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007400d1e18> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
      	- locked <0x00000007400d1e18> (a java.lang.ref.ReferenceQueue$Lock)
      	at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "RMI Scheduler(0)" #19 daemon prio=5 os_prio=31 tid=0x00007fe60c2c4000 nid=0x7d03 waiting on condition [0x000070000662b000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x000000074017e5b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "GC Daemon" #17 daemon prio=2 os_prio=31 tid=0x00007fe60c2bb000 nid=0x7903 in Object.wait() [0x0000700006425000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007402b00c8> (a sun.misc.GC$LatencyLock)
      	at sun.misc.GC$Daemon.run(GC.java:117)
      	- locked <0x00000007402b00c8> (a sun.misc.GC$LatencyLock)
      
         Locked ownable synchronizers:
      	- None
      
      "RMI Reaper" #16 prio=5 os_prio=31 tid=0x00007fe60d001800 nid=0x7703 in Object.wait() [0x0000700006322000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007401d9310> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
      	- locked <0x00000007401d9310> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
      	at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:351)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "RMI TCP Accept-0" #15 daemon prio=5 os_prio=31 tid=0x00007fe60a902000 nid=0x7503 runnable [0x000070000621f000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.PlainSocketImpl.socketAccept(Native Method)
      	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
      	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
      	at java.net.ServerSocket.accept(ServerSocket.java:513)
      	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:389)
      	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:361)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "RMI TCP Accept-1099" #14 daemon prio=5 os_prio=31 tid=0x00007fe60a411800 nid=0x7303 runnable [0x000070000611c000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.PlainSocketImpl.socketAccept(Native Method)
      	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
      	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
      	at java.net.ServerSocket.accept(ServerSocket.java:513)
      	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:389)
      	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:361)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "ActiveMQ Artemis Server Shutdown Timer" #12 daemon prio=5 os_prio=31 tid=0x00007fe60b80a800 nid=0x7103 in Object.wait() [0x0000700006019000]
         java.lang.Thread.State: TIMED_WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	at java.util.TimerThread.mainLoop(Timer.java:552)
      	- locked <0x00000007402450b8> (a java.util.TaskQueue)
      	at java.util.TimerThread.run(Timer.java:505)
      
         Locked ownable synchronizers:
      	- None
      
      "Service Thread" #10 daemon prio=9 os_prio=31 tid=0x00007fe60916e000 nid=0x6d03 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C1 CompilerThread3" #9 daemon prio=9 os_prio=31 tid=0x00007fe60a02d800 nid=0x6b03 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C2 CompilerThread2" #8 daemon prio=9 os_prio=31 tid=0x00007fe609812800 nid=0x6903 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C2 CompilerThread1" #7 daemon prio=9 os_prio=31 tid=0x00007fe609163000 nid=0x6703 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C2 CompilerThread0" #6 daemon prio=9 os_prio=31 tid=0x00007fe60c016800 nid=0x6503 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Signal Dispatcher" #5 daemon prio=9 os_prio=31 tid=0x00007fe60c014800 nid=0x6303 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=31 tid=0x00007fe609158800 nid=0x6003 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007fe609152800 nid=0x5103 in Object.wait() [0x00007000056fe000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007401972f8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
      	- locked <0x00000007401972f8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
      
         Locked ownable synchronizers:
      	- None
      
      "Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007fe60a02a800 nid=0x4f03 in Object.wait() [0x00007000055fb000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x0000000740348588> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:502)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
      	- locked <0x0000000740348588> (a java.lang.ref.Reference$Lock)
      
         Locked ownable synchronizers:
      	- None
      
      "VM Thread" os_prio=31 tid=0x00007fe60a028000 nid=0x4d03 runnable 
      
      "Gang worker#0 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60a00e800 nid=0x2503 runnable 
      
      "Gang worker#1 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60900c800 nid=0x2703 runnable 
      
      "Gang worker#2 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60a00f000 nid=0x2903 runnable 
      
      "Gang worker#3 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60900d000 nid=0x2b03 runnable 
      
      "Gang worker#4 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60a010000 nid=0x2d03 runnable 
      
      "Gang worker#5 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60900e000 nid=0x2f03 runnable 
      
      "Gang worker#6 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60a800000 nid=0x3103 runnable 
      
      "Gang worker#7 (Parallel GC Threads)" os_prio=31 tid=0x00007fe60a010800 nid=0x3303 runnable 
      
      "G1 Main Concurrent Mark GC Thread" os_prio=31 tid=0x00007fe60980a000 nid=0x4703 runnable 
      
      "Gang worker#0 (G1 Parallel Marking Threads)" os_prio=31 tid=0x00007fe609010000 nid=0x4903 runnable 
      
      "Gang worker#1 (G1 Parallel Marking Threads)" os_prio=31 tid=0x00007fe609010800 nid=0x4b03 runnable 
      
      "G1 Concurrent Refinement Thread#0" os_prio=31 tid=0x00007fe60a012000 nid=0x4503 runnable 
      
      "G1 Concurrent Refinement Thread#1" os_prio=31 tid=0x00007fe60b801800 nid=0x4303 runnable 
      
      "G1 Concurrent Refinement Thread#2" os_prio=31 tid=0x00007fe60a011000 nid=0x4103 runnable 
      
      "G1 Concurrent Refinement Thread#3" os_prio=31 tid=0x00007fe60900f000 nid=0x3f03 runnable 
      
      "G1 Concurrent Refinement Thread#4" os_prio=31 tid=0x00007fe60b801000 nid=0x3d03 runnable 
      
      "G1 Concurrent Refinement Thread#5" os_prio=31 tid=0x00007fe60900e800 nid=0x3b03 runnable 
      
      "G1 Concurrent Refinement Thread#6" os_prio=31 tid=0x00007fe60b800000 nid=0x3903 runnable 
      
      "G1 Concurrent Refinement Thread#7" os_prio=31 tid=0x00007fe609802000 nid=0x3703 runnable 
      
      "G1 Concurrent Refinement Thread#8" os_prio=31 tid=0x00007fe609801000 nid=0x3503 runnable 
      
      "VM Periodic Task Thread" os_prio=31 tid=0x00007fe60915e800 nid=0x6f03 waiting on condition 
      
      JNI global references: 294
      

      Attachments

        Issue Links

          Activity

            People

              jbertram Justin Bertram
              jbertram Justin Bertram
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: