Uploaded image for project: 'Sling'
  1. Sling
  2. SLING-4755

DiscoveryService isn't shutdown aware (Asynchronously sending topology events)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • Discovery Impl 1.0.10
    • Discovery Impl 1.1.6
    • Extensions
    • None
    • AEM 6 SP2

    Description

      DiscoveryServiceImpl seems to perform write operations and require repository reference during a shutdown and it blocks the shutdown for a unpredictable time(here, almost 26 minutes).

      Error log

      26.05.2015 16:54:19.249 *INFO* [Thread-60] org.apache.sling.discovery.impl Service [org.apache.sling.discovery.impl.DiscoveryServiceImpl,4131] ServiceEvent UNREGISTERING
      26.05.2015 16:55:20.756 *INFO* [pool-6-thread-1] com.adobe.granite.repository Service [5185] ServiceEvent REGISTERED
      26.05.2015 17:21:40.298 *INFO* [FelixStartLevel] com.adobe.granite.repository Service [5185] ServiceEvent UNREGISTERING
      

      Thread dump for Thread-60"

      Thread-60" daemon prio=10 tid=0x00007f773c5b8000 nid=0x26e1 runnable [0x00007f783b0ea000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketOutputStream.socketWrite0(Native Method)
      	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
      	at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
      	at org.bson.io.PoolOutputBuffer.pipe(PoolOutputBuffer.java:153)
      	at com.mongodb.OutMessage.pipe(OutMessage.java:236)
      	at com.mongodb.DBPort$1.execute(DBPort.java:140)
      	at com.mongodb.DBPort$1.execute(DBPort.java:135)
      	at com.mongodb.DBPort.doOperation(DBPort.java:164)
      	- locked <0x000000050a8eee30> (a com.mongodb.DBPort)
      	at com.mongodb.DBPort.call(DBPort.java:135)
      	at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:292)
      	at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:271)
      	at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)
      	at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
      	at com.mongodb.DBCursor._check(DBCursor.java:458)
      	at com.mongodb.DBCursor._hasNext(DBCursor.java:546)
      	at com.mongodb.DBCursor.hasNext(DBCursor.java:571)
      	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:500)
      	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:437)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.readChildDocs(DocumentNodeStore.java:906)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.readChildren(DocumentNodeStore.java:843)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$4.call(DocumentNodeStore.java:799)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$4.call(DocumentNodeStore.java:796)
      	at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
      	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
      	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
      	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
      	- locked <0x0000000768234398> (a com.google.common.cache.LocalCache$StrongAccessEntry)
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
      	at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
      	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.getChildren(DocumentNodeStore.java:796)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeState.getChildNodeCount(DocumentNodeState.java:183)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.getChildNodeCount(ModifiedNodeState.java:198)
      	at org.apache.jackrabbit.oak.plugins.memory.MutableNodeState.getChildNodeCount(MutableNodeState.java:265)
      	at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.getChildNodeCount(MemoryNodeBuilder.java:293)
      	at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy.prune(ContentMirrorStoreStrategy.java:464)
      	at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy.remove(ContentMirrorStoreStrategy.java:110)
      	at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy.update(ContentMirrorStoreStrategy.java:84)
      	at org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.leave(PropertyIndexEditor.java:261)
      	at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
      	at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeDeleted(EditorDiff.java:176)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:389)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
      	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
      	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
      	at org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
      	at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
      	at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
      	at org.apache.jackrabbit.oak.spi.state.AbstractNodeStoreBranch$InMemory.merge(AbstractNodeStoreBranch.java:535)
      	at org.apache.jackrabbit.oak.spi.state.AbstractNodeStoreBranch.merge(AbstractNodeStoreBranch.java:330)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge(DocumentNodeStoreBranch.java:148)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentRootBuilder.merge(DocumentRootBuilder.java:159)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.merge(DocumentNodeStore.java:1468)
      	at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:247)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:391)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:537)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.perform(SessionImpl.java:417)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.perform(SessionImpl.java:414)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:294)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl.perform(SessionImpl.java:127)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:414)
      	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.sling.jcr.base.SessionProxyHandler$SessionProxyInvocationHandler.invoke(SessionProxyHandler.java:113)
      	at com.sun.proxy.$Proxy7.save(Unknown Source)
      	at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProvider.commit(JcrResourceProvider.java:522)
      	at org.apache.sling.resourceresolver.impl.helper.ResourceResolverContext.commit(ResourceResolverContext.java:171)
      	at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.commit(ResourceResolverImpl.java:1118)
      	at org.apache.sling.event.impl.jobs.tasks.CheckTopologyTask$2.handle(CheckTopologyTask.java:257)
      	at org.apache.sling.event.impl.jobs.JobTopicTraverser.traverse(JobTopicTraverser.java:145)
      	at org.apache.sling.event.impl.jobs.JobTopicTraverser.traverse(JobTopicTraverser.java:102)
      	at org.apache.sling.event.impl.jobs.tasks.CheckTopologyTask.assignJobs(CheckTopologyTask.java:240)
      	at org.apache.sling.event.impl.jobs.tasks.CheckTopologyTask.reassignJobsFromStoppedInstances(CheckTopologyTask.java:90)
      	at org.apache.sling.event.impl.jobs.tasks.CheckTopologyTask.fullRun(CheckTopologyTask.java:318)
      	at org.apache.sling.event.impl.jobs.config.JobManagerConfiguration.startProcessing(JobManagerConfiguration.java:492)
      	at org.apache.sling.event.impl.jobs.config.JobManagerConfiguration.handleTopologyEvent(JobManagerConfiguration.java:550)
      	- locked <0x0000000519d51778> (a java.util.ArrayList)
      	at org.apache.sling.discovery.impl.DiscoveryServiceImpl.sendTopologyEvent(DiscoveryServiceImpl.java:192)
      	at org.apache.sling.discovery.impl.DiscoveryServiceImpl.handlePotentialTopologyChange(DiscoveryServiceImpl.java:484)
      	at org.apache.sling.discovery.impl.DiscoveryServiceImpl.bindPropertyProviderInteral(DiscoveryServiceImpl.java:272)
      	at org.apache.sling.discovery.impl.DiscoveryServiceImpl.updatedPropertyProvider(DiscoveryServiceImpl.java:285)
      	- locked <0x0000000519d6d068> (a java.lang.Object)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
      	at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
      	at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
      	at org.apache.felix.scr.impl.helper.BindMethod.invoke(BindMethod.java:41)
      	at org.apache.felix.scr.impl.manager.DependencyManager.invokeUpdatedMethod(DependencyManager.java:1655)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeUpdatedMethod(SingleComponentManager.java:381)
      	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.modifiedService(DependencyManager.java:357)
      	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.modifiedService(DependencyManager.java:303)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerModified(ServiceTracker.java:1505)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerModified(ServiceTracker.java:1413)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.track(ServiceTracker.java:1161)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1444)
      	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:940)
      	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
      	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
      	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4425)
      	at org.apache.felix.framework.Felix.access$000(Felix.java:75)
      	at org.apache.felix.framework.Felix$1.serviceChanged(Felix.java:402)
      	at org.apache.felix.framework.ServiceRegistry.servicePropertiesModified(ServiceRegistry.java:503)
      	at org.apache.felix.framework.ServiceRegistrationImpl.setProperties(ServiceRegistrationImpl.java:115)
      	at org.apache.sling.event.impl.jobs.JobConsumerManager.unbindService(JobConsumerManager.java:364)
      	at org.apache.sling.event.impl.jobs.JobConsumerManager.unbindJobConsumer(JobConsumerManager.java:262)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
      	at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
      	at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
      	at org.apache.felix.scr.impl.helper.BindMethod.invoke(BindMethod.java:41)
      	at org.apache.felix.scr.impl.manager.DependencyManager.invokeUnbindMethod(DependencyManager.java:1721)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeUnbindMethod(SingleComponentManager.java:392)
      	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.removedService(DependencyManager.java:372)
      	at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.removedService(DependencyManager.java:303)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1518)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1413)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.untrack(ServiceTracker.java:1273)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1452)
      	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:940)
      	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
      	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
      	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4425)
      	at org.apache.felix.framework.Felix.access$000(Felix.java:75)
      	at org.apache.felix.framework.Felix$1.serviceChanged(Felix.java:402)
      	at org.apache.felix.framework.ServiceRegistry.unregisterService(ServiceRegistry.java:153)
      	at org.apache.felix.framework.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:128)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.unregister(AbstractComponentManager.java:883)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.unregister(AbstractComponentManager.java:857)
      	at org.apache.felix.scr.impl.manager.RegistrationManager.changeRegistration(RegistrationManager.java:140)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager.unregisterService(AbstractComponentManager.java:925)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager.doDeactivate(AbstractComponentManager.java:774)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager.deactivateInternal(AbstractComponentManager.java:757)
      	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:970)
      	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:891)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1518)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1413)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.untrack(ServiceTracker.java:1273)
      	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1452)
      	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:940)
      	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
      	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
      	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4425)
      	at org.apache.felix.framework.Felix.access$000(Felix.java:75)
      	at org.apache.felix.framework.Felix$1.serviceChanged(Felix.java:402)
      	at org.apache.felix.framework.ServiceRegistry.unregisterService(ServiceRegistry.java:153)
      	at org.apache.felix.framework.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:128)
      	at org.apache.sling.resourceresolver.impl.ResourceResolverFactoryActivator.unregisterFactory(ResourceResolverFactoryActivator.java:562)
      	at org.apache.sling.resourceresolver.impl.ResourceResolverFactoryActivator.unregisterFactory(ResourceResolverFactoryActivator.java:553)
      	at org.apache.sling.resourceresolver.impl.ResourceResolverFactoryActivator.run(ResourceResolverFactoryActivator.java:686)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- <0x0000000502653dd0> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
      	- <0x0000000535b59040> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
      

      Attachments

        Issue Links

          Activity

            People

              stefanegli Stefan Egli
              tuerkben Ilyas T├╝rkben
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: