[KARAF-2046] Deadlock on console refresh - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Cannot Reproduce
Affects Version/s: 2.3.0
Fix Version/s: None
Component/s: karaf
Labels:
None

Description

There is a dead/live lock problem when the 2.3.0 karaf console bundle is refreshed. This can be reproduced by refreshing the console bundle:

        __ __                  ____
       / //_/____ __________ _/ __/
      / ,<  / __ `/ ___/ __ `/ /_
     / /| |/ /_/ / /  / /_/ / __/
    /_/ |_|\__,_/_/   \__,_/_/

  Apache Karaf (2.3.0)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or 'osgi:shutdown' to shutdown Karaf.

karaf@root> list -t 0 | grep -i console
[  14] [Active     ] [Created     ] [   25] Apache Karaf :: Shell :: Console (2.
3.0)
karaf@root> refresh 14
You are about to access system bundle 14.  Do you wish to continue (yes/no): kar
af@root>

Do not press any other key and take a thread dump. There will be a FelixFrameworkWiring thread waiting to acquire the NonBlockingInputStream lock and a Karaf Shell Console Thread holding that lock and waiting for input:

"FelixFrameworkWiring" daemon prio=6 tid=0x000000000a582800 nid=0x1cb4 waiting for monitor entry [0x000000000f26d000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at jline.internal.NonBlockingInputStream.shutdown(NonBlockingInputStream.java:75)
	- waiting to lock <0x00000000e0096d48> (a jline.internal.NonBlockingInputStream)
	at jline.console.ConsoleReader.shutdown(ConsoleReader.java:281)
	at org.apache.karaf.shell.console.jline.Console.close(Console.java:149)
	at org.apache.karaf.shell.console.jline.ConsoleFactory.stop(ConsoleFactory.java:196)
	at org.apache.karaf.shell.console.jline.ConsoleFactory.unregisterCommandProcessor(ConsoleFactory.java:70)
	- locked <0x00000000e0d2e758> (a org.apache.karaf.shell.console.jline.ConsoleFactory)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:297)
	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe$Listener.invokeMethods(AbstractServiceReferenceRecipe.java:526)
	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe$Listener.unbind(AbstractServiceReferenceRecipe.java:512)
	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe.unbind(AbstractServiceReferenceRecipe.java:415)
	at org.apache.aries.blueprint.container.ReferenceRecipe.unbind(ReferenceRecipe.java:181)
	- locked <0x00000000e00e7268> (a java.lang.Object)
	at org.apache.aries.blueprint.container.ReferenceRecipe.doStop(ReferenceRecipe.java:122)
	- locked <0x00000000e00e7268> (a java.lang.Object)
	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe.stop(AbstractServiceReferenceRecipe.java:153)
	- locked <0x00000000e00e71f8> (a java.util.ArrayList)
	at org.apache.aries.blueprint.container.BlueprintContainerImpl.untrackServiceReference(BlueprintContainerImpl.java:594)
	at org.apache.aries.blueprint.container.BlueprintContainerImpl.untrackServiceReferences(BlueprintContainerImpl.java:574)
	at org.apache.aries.blueprint.container.BlueprintContainerImpl.destroy(BlueprintContainerImpl.java:842)
	at org.apache.aries.blueprint.container.BlueprintExtender.destroyContext(BlueprintExtender.java:246)
	at org.apache.aries.blueprint.container.BlueprintExtender.bundleChanged(BlueprintExtender.java:238)
	at org.apache.aries.blueprint.container.BlueprintExtender$BlueprintBundleTrackerCustomizer.modifiedBundle(BlueprintExtender.java:434)
	at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:198)
	at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:128)
	at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:468)
	at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:161)
	at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEventHook.event(BundleHookBundleTracker.java:117)
	at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1103)
	at org.apache.felix.framework.util.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:695)
	at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:483)
	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4244)
	at org.apache.felix.framework.Felix.stopBundle(Felix.java:2351)
	at org.apache.felix.framework.Felix$RefreshHelper.stop(Felix.java:4629)
	at org.apache.felix.framework.Felix.refreshPackages(Felix.java:3951)
	at org.apache.felix.framework.FrameworkWiringImpl.run(FrameworkWiringImpl.java:172)
	at java.lang.Thread.run(Thread.java:722)

"Karaf Shell Console Thread" daemon prio=6 tid=0x000000000a574800 nid=0x1d08 waiting on condition [0x000000000dcfe000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000e00e6d38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
	at org.apache.karaf.shell.console.jline.Console$ConsoleInputStream.read(Console.java:386)
	at org.apache.karaf.shell.console.jline.Console$ConsoleInputStream.read(Console.java:403)
	at jline.internal.NonBlockingInputStream.read(NonBlockingInputStream.java:169)
	- locked <0x00000000e0096d48> (a jline.internal.NonBlockingInputStream)
	at jline.internal.NonBlockingInputStream.read(NonBlockingInputStream.java:137)
	at jline.internal.NonBlockingInputStream.read(NonBlockingInputStream.java:246)
	at jline.internal.InputStreamReader.read(InputStreamReader.java:259)
	- locked <0x00000000e0096d48> (a jline.internal.NonBlockingInputStream)
	at jline.internal.InputStreamReader.read(InputStreamReader.java:196)
	- locked <0x00000000e0096d48> (a jline.internal.NonBlockingInputStream)
	at jline.console.ConsoleReader.readCharacter(ConsoleReader.java:1974)
	at jline.console.ConsoleReader.readLine(ConsoleReader.java:2174)
	at jline.console.ConsoleReader.readLine(ConsoleReader.java:2098)
	at org.apache.karaf.shell.console.jline.Console.readAndParseCommand(Console.java:235)
	at org.apache.karaf.shell.console.jline.Console.run(Console.java:171)
	at java.lang.Thread.run(Thread.java:722)

No progress will be made until a key is pressed. After pressing a key the error message "Error executing command: read() with timeout cannot be called as non-blocking operation is disabled" will be printed and the Apache Karaf 2.3.0 logo will be printed a second time.

This is particularly problematic from environments such as pax exam where no further console input can be generated. Additionally a refresh on the console bundle seems to be generated sometimes when installing the jpa/jndi features which include org.apache.aries.util as a dependency. See this karaf-user thread for more information:

http://karaf.922171.n3.nabble.com/Problems-with-ipojo-in-Karaf-2-3-0-td4026688.html

Deadlock on console refresh

Details

Description

Attachments

Activity

People

Dates