Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.5
    • Fix Version/s: 2.3.0, 3.0.0
    • Component/s: karaf-shell
    • Labels:
      None
    • Environment:

      Ubuntu 11.10 64-bit, OpenJDK 6

      Description

      Example usage:

      echo osgi:ls | bin/client
      

      or:

      bin/client < commands.karafsh
      

      or redirecting stdout to file:

      echo osgi:ls | bin/client > bundles.txt
      

      Expected outcome:
      1. Output of the command printed to stdout
      2. Banners etc. (if any) printed to stderr
      3. Shell prompt should not be output at all
      4. The client exits immediately

      Actual behavior:
      1. All output including banners and the shell prompt are printed to stdout (only)
      2. The client then hangs but still using around 20% CPU usage. It's not possible to Ctrl+C, and need to kill the process.

      GNU tools (like bash) tend to detect the presence of an input stream, to differ between interactive mode and a batch mode.
      If this is not possible, can also add a switch e.g. "--batch" or "-b" for "Batch mode".

      Note that the karaf "bin/shell" script already handles this quite nicely... by not hanging after executing commands, and exits properly.

      BTW this should also be possible to do on karaf launcher itself (i.e. start Karaf, execute command(s), then immediately shutdown the runtime).

      1. KARAF-1226-shell.patch
        8 kB
        Hendy Irawan
      2. KARAF-1226-client.patch
        4 kB
        Hendy Irawan
      3. KARAF-1226-main.patch
        13 kB
        Hendy Irawan

        Activity

        Hide
        Hendy Irawan added a comment -
        Show
        Hendy Irawan added a comment - It's possible to detect a tty/terminal stdin using C : http://stackoverflow.com/questions/1312922/detect-if-stdin-is-a-terminal-or-pipe-in-c-c-qt
        Hide
        Hendy Irawan added a comment -

        PATCH: Allow bin/shell script to process several commands in batch mode using -b or --batch switch.

        Show
        Hendy Irawan added a comment - PATCH: Allow bin/shell script to process several commands in batch mode using -b or --batch switch.
        Hide
        Hendy Irawan added a comment -

        Batch mode for client. It works!

        Show
        Hendy Irawan added a comment - Batch mode for client. It works!
        Hide
        Hendy Irawan added a comment -

        Please apply both patches!

        I will be very happy if this can make it into Karaf 2.2.6 (and eventually into ServiceMix 4.4.2 / 4.5)

        Show
        Hendy Irawan added a comment - Please apply both patches! I will be very happy if this can make it into Karaf 2.2.6 (and eventually into ServiceMix 4.4.2 / 4.5)
        Hide
        Jean-Baptiste Onofré added a comment -

        Thanks for the patches, I gonna review it.

        Show
        Jean-Baptiste Onofré added a comment - Thanks for the patches, I gonna review it.
        Hide
        Hendy Irawan added a comment -

        Thank you Jean-Baptiste.

        I'm also attempting to make it into Karaf primary launcher. The way it works is:
        1. If the execute command or batch mode is enabled, it will start a thread that will monitor bundles. Right now I'm using a periodic sleep (~100ms?) to check bundles, but it can be optimized to only listen for bundle state change events.
        2. Once all bundles are "stable" (i.e. nobody is STARTING or STOPPING), then it executes the command(s), and shutdowns the runtime.

        I still don't know how to bypass the branding and the prompt. i.e. I'd like to execute the launcher in "exec" mode, not in interactive mode. Can you give me pointers ?

        Since this will use args (which are currently ignored by Karaf, perhaps so that args can be used fully by application integrator/downstream e.g. ServiceMix), I'll make it configurable, perhaps adding a System property that is enabled by default, but can be disabled by e.g. ServiceMix.

        However I'd be interested if later, this functionality can also be used by ServiceMix, i.e. it should be possible for ServiceMix to process args, while retaining the functionality of exec/batch mode built-in Karaf.

        Show
        Hendy Irawan added a comment - Thank you Jean-Baptiste. I'm also attempting to make it into Karaf primary launcher. The way it works is: 1. If the execute command or batch mode is enabled, it will start a thread that will monitor bundles. Right now I'm using a periodic sleep (~100ms?) to check bundles, but it can be optimized to only listen for bundle state change events. 2. Once all bundles are "stable" (i.e. nobody is STARTING or STOPPING), then it executes the command(s), and shutdowns the runtime. I still don't know how to bypass the branding and the prompt. i.e. I'd like to execute the launcher in "exec" mode, not in interactive mode. Can you give me pointers ? Since this will use args (which are currently ignored by Karaf, perhaps so that args can be used fully by application integrator/downstream e.g. ServiceMix), I'll make it configurable, perhaps adding a System property that is enabled by default, but can be disabled by e.g. ServiceMix. However I'd be interested if later, this functionality can also be used by ServiceMix, i.e. it should be possible for ServiceMix to process args, while retaining the functionality of exec/batch mode built-in Karaf.
        Hide
        Hendy Irawan added a comment -

        some info to self for better understanding the main Karaf launcher :

        Description Resource Path Location
        executing initialization script Console.java /org.apache.karaf.shell.console/src/main/java/org/apache/karaf/shell/console/jline line 173
        creates the Karaf Shell Console Thread ConsoleFactory.java /org.apache.karaf.shell.console/src/main/java/org/apache/karaf/shell/console/jline line 115

        Show
        Hendy Irawan added a comment - some info to self for better understanding the main Karaf launcher : Description Resource Path Location executing initialization script Console.java /org.apache.karaf.shell.console/src/main/java/org/apache/karaf/shell/console/jline line 173 creates the Karaf Shell Console Thread ConsoleFactory.java /org.apache.karaf.shell.console/src/main/java/org/apache/karaf/shell/console/jline line 115
        Hide
        Hendy Irawan added a comment -

        Note: FuseSource already implemented the isatty function, which takes a file descriptor number (org.fusesource.jansi.AnsiConsole.STDOUT_FILENO = 1)

        int org.fusesource.jansi.internal.CLibrary.isatty(int arg0)

        This should make it possible for bin/shell and bin/client to not require the '-b' switch and enable batch mode based on tty state.

        Show
        Hendy Irawan added a comment - Note: FuseSource already implemented the isatty function, which takes a file descriptor number (org.fusesource.jansi.AnsiConsole.STDOUT_FILENO = 1) int org.fusesource.jansi.internal.CLibrary.isatty(int arg0) This should make it possible for bin/shell and bin/client to not require the '-b' switch and enable batch mode based on tty state.
        Hide
        Hendy Irawan added a comment -

        Some more notes to self:

        The OSGi service interface needed to execute commands is org.apache.felix.service.command.CommandProcessor, provided by mvn:org.apache.karaf.shell/org.apache.karaf.shell.console/2.2.5
        "org.apache.felix.service.command.CommandSession" is never registered as OSGi services

        Useful Integration test :
        ./itests/tests/src/test/java/org/apache/karaf/shell/itests/CoreTest.java:

                CommandProcessor cp = getOsgiService(CommandProcessor.class);
                CommandSession cs = cp.createSession(System.in, System.out, System.err);
        
                try {
                    cs.execute("log:display");
                    fail("command should not exist");
                } catch (IllegalArgumentException e) {
                    assertTrue(e.getMessage().indexOf("Command not found") >= 0);
                }
        
                Bundle b = getInstalledBundle("org.apache.karaf.shell.log");
                b.start();
        
                Thread.sleep(1000);
        
                cs.execute("log:display");
        
                b.stop();
        
                Thread.sleep(1000);
        
                try {
                    cs.execute("log:display");
                    fail("command should not exist");
                } catch (IllegalArgumentException e) {
                    assertTrue(e.getMessage().indexOf("Command not found") >= 0);
                }
        
                cs.close();
        

        getOsgiService() implementation in tooling/testing/src/main/java/org/apache/karaf/testing/AbstractIntegrationTest.java :

        Get the bundle which provide a service :

        karaf@root> service:list | grep -B 5 CommandProcessor
        org.osgi.service.blueprint.container.BlueprintContainer
        
        Apache Karaf :: Shell :: Console (14) provides:
        -----------------------------------------------
        org.apache.felix.service.threadio.ThreadIO
        org.apache.felix.service.command.CommandProcessor
        

        Then use osgi:info 14 or osgi:headers 14

        Lifecycle
        1. ConsoleFactory is created in shell/console/src/main/resources/OSGI-INF/blueprint/karaf-console.xml
        2. ConsoleFactory creates Console ONLY if karaf.startLocalConsole is true (default is true).
        3. Console constructor creates CommandSession plus the whole unnecessary shebang

        Example of karaf.startLocalConsole = false :
        1. demos/web/src/main/java/org/apache/karaf/web/WebAppListener.java
        2. admin/core/src/main/java/org/apache/karaf/admin/internal/InstanceImpl.java: + " -Dkaraf.startLocalConsole=false"
        3. tooling/testing/src/main/java/org/apache/karaf/testing/Helper.java: sysProps.setProperty("karaf.startLocalConsole", "false");

        Also want to set system property karaf.startRemoteShell = false to reduce unnecessary overhead.

        Show
        Hendy Irawan added a comment - Some more notes to self: The OSGi service interface needed to execute commands is org.apache.felix.service.command.CommandProcessor, provided by mvn:org.apache.karaf.shell/org.apache.karaf.shell.console/2.2.5 "org.apache.felix.service.command.CommandSession" is never registered as OSGi services Useful Integration test : ./itests/tests/src/test/java/org/apache/karaf/shell/itests/CoreTest.java: CommandProcessor cp = getOsgiService(CommandProcessor.class); CommandSession cs = cp.createSession( System .in, System .out, System .err); try { cs.execute( "log:display" ); fail( "command should not exist" ); } catch (IllegalArgumentException e) { assertTrue(e.getMessage().indexOf( "Command not found" ) >= 0); } Bundle b = getInstalledBundle( "org.apache.karaf.shell.log" ); b.start(); Thread .sleep(1000); cs.execute( "log:display" ); b.stop(); Thread .sleep(1000); try { cs.execute( "log:display" ); fail( "command should not exist" ); } catch (IllegalArgumentException e) { assertTrue(e.getMessage().indexOf( "Command not found" ) >= 0); } cs.close(); getOsgiService() implementation in tooling/testing/src/main/java/org/apache/karaf/testing/AbstractIntegrationTest.java : Get the bundle which provide a service : karaf@root> service:list | grep -B 5 CommandProcessor org.osgi.service.blueprint.container.BlueprintContainer Apache Karaf :: Shell :: Console (14) provides: ----------------------------------------------- org.apache.felix.service.threadio.ThreadIO org.apache.felix.service.command.CommandProcessor Then use osgi:info 14 or osgi:headers 14 Lifecycle 1. ConsoleFactory is created in shell/console/src/main/resources/OSGI-INF/blueprint/karaf-console.xml 2. ConsoleFactory creates Console ONLY if karaf.startLocalConsole is true (default is true). 3. Console constructor creates CommandSession plus the whole unnecessary shebang Example of karaf.startLocalConsole = false : 1. demos/web/src/main/java/org/apache/karaf/web/WebAppListener.java 2. admin/core/src/main/java/org/apache/karaf/admin/internal/InstanceImpl.java: + " -Dkaraf.startLocalConsole=false" 3. tooling/testing/src/main/java/org/apache/karaf/testing/Helper.java: sysProps.setProperty("karaf.startLocalConsole", "false"); Also want to set system property karaf.startRemoteShell = false to reduce unnecessary overhead.
        Hide
        Hendy Irawan added a comment -

        I can make command execution in the Karaf launcher work, however... I'm having difficulty how to detect when the entire Karaf runtime is "stable".

        I used OSGi bundle, service, and Blueprint event listeners combined with signaling timeout. However, timeouts lower than 100ms sometimes cause a race condition, i.e. no event was generated within 75ms thus the command is executed, however the services required by the commands are actually not yet registered. The dilemma is than higher timeouts will cause higher latency, thus giving the impression that executing the command is slow, while in fact it was spent waiting.

        Any suggestions?

        Show
        Hendy Irawan added a comment - I can make command execution in the Karaf launcher work, however... I'm having difficulty how to detect when the entire Karaf runtime is "stable". I used OSGi bundle, service, and Blueprint event listeners combined with signaling timeout. However, timeouts lower than 100ms sometimes cause a race condition, i.e. no event was generated within 75ms thus the command is executed, however the services required by the commands are actually not yet registered. The dilemma is than higher timeouts will cause higher latency, thus giving the impression that executing the command is slow, while in fact it was spent waiting. Any suggestions?
        Hide
        Hendy Irawan added a comment -

        Woohoo. Right now this is working :

        ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf 'ls | grep karaf'
        org.apache.karaf.features.FeaturesService
        org.apache.karaf.jaas.config.KeystoreManager
        org.apache.karaf.admin.AdminService
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.features.FeaturesListener
        org.apache.karaf.jaas.modules.EncryptionService
        org.apache.karaf.jaas.config.JaasRealm
        org.apache.karaf.jaas.modules.BackingEngineFactory
        
        ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf 'ls | grep karaf' > ~/tmp/karaf.txt
        
        ceefour@annafi:~/apache-karaf-2.2.5$ cat ~/tmp/karaf.txt 
        org.apache.karaf.features.FeaturesService
        org.apache.karaf.jaas.config.KeystoreManager
        org.apache.karaf.admin.AdminService
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.diagnostic.core.DumpProvider
        org.apache.karaf.features.FeaturesListener
        org.apache.karaf.jaas.modules.EncryptionService
        org.apache.karaf.jaas.modules.BackingEngineFactory
        org.apache.karaf.jaas.config.JaasRealm
        

        The built-in TTY detection is already working, the Terminal one has ANSI colors, but when redirected no ANSI colors are used.

        I will add 'c' or '-command=' to execute one or more commands.

        For batch mode, it should detect using isatty(), so should not be required to use a '-b' switch.

        Show
        Hendy Irawan added a comment - Woohoo. Right now this is working : ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf 'ls | grep karaf' org.apache.karaf.features.FeaturesService org.apache.karaf.jaas.config.KeystoreManager org.apache.karaf.admin.AdminService org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.features.FeaturesListener org.apache.karaf.jaas.modules.EncryptionService org.apache.karaf.jaas.config.JaasRealm org.apache.karaf.jaas.modules.BackingEngineFactory ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf 'ls | grep karaf' > ~/tmp/karaf.txt ceefour@annafi:~/apache-karaf-2.2.5$ cat ~/tmp/karaf.txt org.apache.karaf.features.FeaturesService org.apache.karaf.jaas.config.KeystoreManager org.apache.karaf.admin.AdminService org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.diagnostic.core.DumpProvider org.apache.karaf.features.FeaturesListener org.apache.karaf.jaas.modules.EncryptionService org.apache.karaf.jaas.modules.BackingEngineFactory org.apache.karaf.jaas.config.JaasRealm The built-in TTY detection is already working, the Terminal one has ANSI colors, but when redirected no ANSI colors are used. I will add ' c' or ' -command=' to execute one or more commands. For batch mode, it should detect using isatty(), so should not be required to use a '-b' switch.
        Hide
        Hendy Irawan added a comment -

        Alright, it's working! with batch mode support

        ceefour@annafi:~/apache-karaf-2.2.5$ echo 'echo hai' | bin/karaf -b
        hai
        
        ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf -c 'echo hello world'
        hello world
        
        Show
        Hendy Irawan added a comment - Alright, it's working! with batch mode support ceefour@annafi:~/apache-karaf-2.2.5$ echo 'echo hai' | bin/karaf -b hai ceefour@annafi:~/apache-karaf-2.2.5$ bin/karaf -c 'echo hello world' hello world
        Hide
        Hendy Irawan added a comment -

        Please apply this patch too.

        With this one, the main launcher ("bin/karaf") also gets support for b/batch and -c/-command support (see my comments above).

        These switches should be documented somewhere.

        Also, I added a configuration property "karaf.args.parse" that is by default enabled. To preserve old behavior (ignoring cmdline args) can also make the default as false. However IMHO it's better to make this default to true (for consistency with bin/client and bin/shell), and modify downstream (i.e. ServiceMix) as appropriate.

        I set the "stabilizing" timeout as 100ms. I can't find a better way to wait for the OSGi runtime to "stabilize". Or perhaps the timeout can be configured in properties?

        Show
        Hendy Irawan added a comment - Please apply this patch too. With this one, the main launcher ("bin/karaf") also gets support for b/ batch and -c/ -command support (see my comments above). These switches should be documented somewhere. Also, I added a configuration property "karaf.args.parse" that is by default enabled. To preserve old behavior (ignoring cmdline args) can also make the default as false. However IMHO it's better to make this default to true (for consistency with bin/client and bin/shell), and modify downstream (i.e. ServiceMix) as appropriate. I set the "stabilizing" timeout as 100ms. I can't find a better way to wait for the OSGi runtime to "stabilize". Or perhaps the timeout can be configured in properties?
        Hide
        Jean-Baptiste Onofré added a comment -

        Thanks for the patches, I'm reviewing it.

        Show
        Jean-Baptiste Onofré added a comment - Thanks for the patches, I'm reviewing it.
        Hide
        Jean-Baptiste Onofré added a comment -

        Move to 2.3.0 as karaf-2.2.x branch is only for bug fixes.

        Show
        Jean-Baptiste Onofré added a comment - Move to 2.3.0 as karaf-2.2.x branch is only for bug fixes.
        Hide
        Hendy Irawan added a comment -

        I've ported this to 2.3.x branch. Please approve.

        https://github.com/soluvas/karaf/tree/2.3.x-KARAF-1226

        I forked using the git.apache.org, so it should be painless to merge.

        Show
        Hendy Irawan added a comment - I've ported this to 2.3.x branch. Please approve. https://github.com/soluvas/karaf/tree/2.3.x-KARAF-1226 I forked using the git.apache.org, so it should be painless to merge.
        Hide
        Hendy Irawan added a comment -

        Tried to port to 3.0 but seems like the codebase has changed dramatically there. Will stick to 2.3.0 for now.

        Just a note (not relevant to this bug), 2.2.9-SNAPSHOT is currently not launching for me for some reason... (some package imports not resolved)

        Show
        Hendy Irawan added a comment - Tried to port to 3.0 but seems like the codebase has changed dramatically there. Will stick to 2.3.0 for now. Just a note (not relevant to this bug), 2.2.9-SNAPSHOT is currently not launching for me for some reason... (some package imports not resolved)
        Hide
        Jean-Baptiste Onofré added a comment -

        Thanks for the update Hendy, and sorry for the delay to review your patches.

        I gonna make a try on 2.2.9-SNAPSHOT by the way.

        Show
        Jean-Baptiste Onofré added a comment - Thanks for the update Hendy, and sorry for the delay to review your patches. I gonna make a try on 2.2.9-SNAPSHOT by the way.
        Hide
        Hendy Irawan added a comment -

        BTW, we periodically publish our "fork" of Karaf here :

        http://nexus.bippo.co.id/nexus/index.html#nexus-search;quick~karaf

        Show
        Hendy Irawan added a comment - BTW, we periodically publish our "fork" of Karaf here : http://nexus.bippo.co.id/nexus/index.html#nexus-search;quick~karaf
        Hide
        Guillaume Nodet added a comment -

        I don't think the launcher idea is a good one. We had that in the first stages, but there's currently no way to detect when things are started. What could be valid for you will be invalid for someone else because of the use of other extenders, so until the OSGi spec comes with something standardized, I don't think we should do anything.

        There are workarounds though. You can always include the needed logic in your own scripts to wait for a given service to become available for example.

        I think including a way to read commands from a file on the client is a really good idea however or to be able to detect that the input stream comes from a file and not real stdin.

        Show
        Guillaume Nodet added a comment - I don't think the launcher idea is a good one. We had that in the first stages, but there's currently no way to detect when things are started. What could be valid for you will be invalid for someone else because of the use of other extenders, so until the OSGi spec comes with something standardized, I don't think we should do anything. There are workarounds though. You can always include the needed logic in your own scripts to wait for a given service to become available for example. I think including a way to read commands from a file on the client is a really good idea however or to be able to detect that the input stream comes from a file and not real stdin.
        Hide
        Guillaume Nodet added a comment -

        Fixed the client and shell scripts on 2.3.x

        Committing to https://svn.apache.org/repos/asf/karaf/branches/karaf-2.3.x ...
        M client/src/main/java/org/apache/karaf/client/Main.java
        M shell/console/src/main/java/org/apache/karaf/shell/console/Main.java
        Committed r1395521

        Show
        Guillaume Nodet added a comment - Fixed the client and shell scripts on 2.3.x Committing to https://svn.apache.org/repos/asf/karaf/branches/karaf-2.3.x ... M client/src/main/java/org/apache/karaf/client/Main.java M shell/console/src/main/java/org/apache/karaf/shell/console/Main.java Committed r1395521
        Hide
        Hendy Irawan added a comment -

        Thank YOU Guillaume !

        Show
        Hendy Irawan added a comment - Thank YOU Guillaume !
        Hide
        Jean-Baptiste Onofré added a comment -

        Fix on karaf-2.3.x: http://svn.apache.org/viewvc?view=revision&revision=1395521

        I gonna do it on trunk as well.

        Show
        Jean-Baptiste Onofré added a comment - Fix on karaf-2.3.x: http://svn.apache.org/viewvc?view=revision&revision=1395521 I gonna do it on trunk as well.
        Show
        Jean-Baptiste Onofré added a comment - Fixed on trunk: http://svn.apache.org/viewvc?view=revision&revision=1396582
        Hide
        Hendy Irawan added a comment -

        Wow, thank you Jean-Baptiste !

        Show
        Hendy Irawan added a comment - Wow, thank you Jean-Baptiste !

          People

          • Assignee:
            Guillaume Nodet
            Reporter:
            Hendy Irawan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development