[FELIX-5776] Memory Leak by Config Admin

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: configadmin-1.8.16
• Fix Version/s: configadmin-1.9.0
• Component/s: Configuration Admin
• Assignee: Carsten Ziegeler
• Reporter: Fabian Lange
• Created: 22/Jan/18 10:29
• Updated: 12/Jun/19 07:50
• Resolved: 12/Jun/19 07:50

Description

I have a heapdump (which I cannot share publicly) of a karaf container which crashed due to out of memory.

The Heapdump shows 30mb occupied by org.apache.felix.cm.impl.CachingPersistenceManagerProxy including several related classes.

I am going to attach a few screenshots from MAT/JProfiler.

There are a few threads involved which look interesting.

 

  Thread "CM Configuration Updater (Delete: pid=ce24c1344-88bb-4c66-bebc-527cc574f347)":
    at sun.misc.Unsafe.park(boolean, long)
    at java.util.concurrent.locks.LockSupport.park(java.lang.Object) (line: 175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() (line: 836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.util.concurrent.locks.AbstractQueuedSynchronizer$Node, int) (line: 870)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(int) (line: 1199)
    at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock() (line: 943)
    at org.apache.felix.cm.impl.CachingPersistenceManagerProxy.store(java.lang.String, java.util.Dictionary) (line: 245)
    at org.apache.felix.cm.impl.Factory.store() (line: 137)
    at org.apache.felix.cm.impl.ConfigurationManager$DeleteConfiguration.run() (line: 1876)
    at org.apache.felix.cm.impl.UpdateThread.run0(java.lang.Runnable) (line: 141)
    at org.apache.felix.cm.impl.UpdateThread.run() (line: 109)
    at java.lang.Thread.run() (line: 748)

  Thread "updater-thread-1":
    at sun.misc.Unsafe.park(boolean, long)
    at java.util.concurrent.locks.LockSupport.park(java.lang.Object) (line: 175)
    at java.util.concurrent.FutureTask.awaitDone(boolean, long) (line: 429)
    at java.util.concurrent.FutureTask.get() (line: 191)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvisionInThread(java.util.Map, java.util.Map, org.apache.karaf.features.internal.service.State, java.util.EnumSet) (line: 1134)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.addRequirements(java.util.Map, java.util.EnumSet) (line: 1077)

  Thread "features-1-thread-1":
    at sun.misc.Unsafe.park(boolean, long)
    at java.util.concurrent.locks.LockSupport.park(java.lang.Object) (line: 175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() (line: 836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(int) (line: 967)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) (line: 1283)
    at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock() (line: 727)
    at org.apache.felix.cm.impl.CachingPersistenceManagerProxy.getDictionaries(org.apache.felix.cm.impl.SimpleFilter) (line: 137)
    at org.apache.felix.cm.impl.ConfigurationManager.listConfigurations(org.apache.felix.cm.impl.ConfigurationAdminImpl, java.lang.String) (line: 660)
    at org.apache.felix.cm.impl.ConfigurationAdminImpl.listConfigurations(java.lang.String) (line: 190)
    at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.findConfigurations(org.osgi.service.cm.ConfigurationAdmin, java.lang.String) (line: 605)
    at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.findFactoryConfigurations(org.osgi.service.cm.ConfigurationAdmin, java.lang.String, org.osgi.framework.Bundle) (line: 540)
    at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configureComponentHolder(org.apache.felix.scr.impl.manager.ComponentHolder) (line: 143)
    at org.apache.felix.scr.impl.BundleComponentActivator.setRegionConfigurationSupport(org.osgi.framework.ServiceReference) (line: 833)
    at org.apache.felix.scr.impl.helper.ConfigAdminTracker$1.addingService(org.osgi.framework.ServiceReference) (line: 71)
    at org.apache.felix.scr.impl.helper.ConfigAdminTracker$1.addingService(org.osgi.framework.ServiceReference) (line: 43)
    at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(org.osgi.framework.ServiceReference, org.osgi.framework.ServiceEvent) (line: 941)
    at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(java.lang.Object, java.lang.Object) (line: 870)
    at org.osgi.util.tracker.AbstractTracked.trackAdding(java.lang.Object, java.lang.Object) (line: 256)
    at org.osgi.util.tracker.AbstractTracked.trackInitial() (line: 183)
    at org.osgi.util.tracker.ServiceTracker.open(boolean) (line: 318)
    at org.osgi.util.tracker.ServiceTracker.open() (line: 261)
    at org.apache.felix.scr.impl.helper.ConfigAdminTracker.<init>(org.apache.felix.scr.impl.manager.ComponentActivator) (line: 88)
    at org.apache.felix.scr.impl.BundleComponentActivator.<init>(org.apache.felix.scr.impl.helper.SimpleLogger, org.apache.felix.scr.impl.ComponentRegistry, org.apache.felix.scr.impl.ComponentActorThread, org.osgi.framework.BundleContext, org.apache.felix.scr.impl.manager.ScrConfiguration) (line: 274)
    at org.apache.felix.scr.impl.Activator.loadComponents(org.osgi.framework.Bundle) (line: 388)
    at org.apache.felix.scr.impl.Activator.access$200(org.apache.felix.scr.impl.Activator, org.osgi.framework.Bundle) (line: 54)
    at org.apache.felix.scr.impl.Activator$ScrExtension.start() (line: 265)
    at org.apache.felix.utils.extender.AbstractExtender.createExtension(org.osgi.framework.Bundle) (line: 254)
    at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(org.osgi.framework.Bundle, org.osgi.framework.BundleEvent, java.lang.Object) (line: 227)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(org.osgi.framework.Bundle, org.osgi.framework.BundleEvent, java.lang.Object) (line: 482)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(java.lang.Object, java.lang.Object, java.lang.Object) (line: 415)
    at org.osgi.util.tracker.AbstractTracked.track(java.lang.Object, java.lang.Object) (line: 232)
    at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(org.osgi.framework.BundleEvent) (line: 444)
    at org.apache.felix.framework.EventDispatcher.invokeBundleListenerCallback(org.osgi.framework.Bundle, java.util.EventListener, java.util.EventObject) (line: 915)
    at org.apache.felix.framework.EventDispatcher.fireEventImmediately(org.apache.felix.framework.EventDispatcher, int, java.util.Map, java.util.EventObject, java.util.Dictionary) (line: 834)
    at org.apache.felix.framework.EventDispatcher.fireBundleEvent(org.osgi.framework.BundleEvent, org.apache.felix.framework.Felix) (line: 516)
    at org.apache.felix.framework.Felix.fireBundleEvent(int, org.osgi.framework.Bundle) (line: 4563)
    at org.apache.felix.framework.Felix.startBundle(org.apache.felix.framework.BundleImpl, int) (line: 2173)
    at org.apache.felix.framework.BundleImpl.start(int) (line: 998)
    at org.apache.felix.framework.BundleImpl.start() (line: 984)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.startBundle(org.osgi.framework.Bundle) (line: 1346)
    at org.apache.karaf.features.internal.service.Deployer.deploy(org.apache.karaf.features.internal.service.Deployer$DeploymentState, org.apache.karaf.features.internal.service.Deployer$DeploymentRequest) (line: 891)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(java.util.Map, java.util.Map, org.apache.karaf.features.internal.service.State, java.util.EnumSet, java.lang.String) (line: 1233)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$0(java.util.Map, java.util.Map, org.apache.karaf.features.internal.service.State, java.util.EnumSet, java.lang.String) (line: 1132)
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl$$Lambda$16.call()
    at java.util.concurrent.FutureTask.run() (line: 266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) (line: 1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run() (line: 617)
    at java.lang.Thread.run() (line: 748)

 

This might as well be a karaf bug.

While it looks like a memory leak, I could also suspect that this might be cause by a concurrency problem, as the threadump shows multiple updating threads.

I can provide more details if required, or share the dump privately.