Commons Exec
  1. Commons Exec
  2. EXEC-50

Pumping lines from the command output

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.1
    • Fix Version/s: 1.2
    • Labels:
      None

      Description

      My application makes heavy use of analyzing output lines of the long running executable. Executed command may run even up to to few hours. Actually I'm not so interested in the result of execution, but rather in the output produced by the program. Each line of the output contains some important data. I need to listen to these lines of the output, filter them and send JMS message containing selected of them.

      Current implementation of the StreamPumper makes it hard to listen to the output lines produced by the executable since it copies entire InputStream. What I did is pumping command output to the FilterInputStream and looking for the new line markers. It works for me, however it would be nice to have an ability to read entire lines from the PumpStream instead of raw bytes only. This is quite common case since executable usually format their output using the lines instead of the raw stream of bytes.

      I created patch refactoring StreamPumper to extend AbstractStreamPumper. I created also BufferedReaderPumper (also extending AbstractStreamPumper) to pump lines from the executable output and send them to the OutputLinesListener. I also extracted creation of the StreamPumper in the PumpStreamHandler to the factory method. This method can be overridden in order to use BufferedReaderPumper instead of default StreamPumper.

      What do you think about such approach?

      1. commons-exec.zip
        398 kB
        Henryk Konsek
      2. lines_listener.patch
        20 kB
        Henryk Konsek

        Activity

        Hide
        Henryk Konsek added a comment -

        Any comments regarding my patch?

        Show
        Henryk Konsek added a comment - Any comments regarding my patch?
        Hide
        Siegfried Goeschl added a comment -

        Hi Henryk,

        I hope I have some time today to look at it more closely .... stay tuned

        Sigi

        Show
        Siegfried Goeschl added a comment - Hi Henryk, I hope I have some time today to look at it more closely .... stay tuned Sigi
        Hide
        Siegfried Goeschl added a comment -

        Added a git repo to play around (https://github.com/sgoeschl/commons-exec)

        Show
        Siegfried Goeschl added a comment - Added a git repo to play around ( https://github.com/sgoeschl/commons-exec )
        Hide
        Siegfried Goeschl added a comment -

        Hi Henryk - I'm not able to properly apply the patch (even when using rev 1022857) - can you send my a source tarball?

        Show
        Siegfried Goeschl added a comment - Hi Henryk - I'm not able to properly apply the patch (even when using rev 1022857) - can you send my a source tarball?
        Hide
        Henryk Konsek added a comment -

        I created previous patch with 'svn diff > patch_file.patch'.

        But I also attach zipped project with local changes. I haven't updated it since the day I submitted the patch so it may be out of sync with the latest trunk.

        You can also send me a Linux command I should use to create patch in the format you want.

        Show
        Henryk Konsek added a comment - I created previous patch with 'svn diff > patch_file.patch'. But I also attach zipped project with local changes. I haven't updated it since the day I submitted the patch so it may be out of sync with the latest trunk. You can also send me a Linux command I should use to create patch in the format you want.
        Hide
        Henryk Konsek added a comment -

        How's the patch? Did you manage to check it out? Maybe you want me to send it in particular format?

        Show
        Henryk Konsek added a comment - How's the patch? Did you manage to check it out? Maybe you want me to send it in particular format?
        Hide
        Siegfried Goeschl added a comment -

        Mhmm, still catching up from my week in Atlante ... ... coming back to the patch during the weekend

        Show
        Siegfried Goeschl added a comment - Mhmm, still catching up from my week in Atlante ... ... coming back to the patch during the weekend
        Hide
        Sebb added a comment -

        Why does LinesListener.patch delete the AL header from the file AbstractStreamPumper.java?
        There also seem to be quite a few spurious changes in the patch for that file

        There needs to be an AL header on the file: OutputLinesListener.java

        Show
        Sebb added a comment - Why does LinesListener.patch delete the AL header from the file AbstractStreamPumper.java? There also seem to be quite a few spurious changes in the patch for that file There needs to be an AL header on the file: OutputLinesListener.java
        Hide
        Siegfried Goeschl added a comment -

        Merged your changes into the trunk and the tests succeed - I will attach a complete patch and review the changes

        Show
        Siegfried Goeschl added a comment - Merged your changes into the trunk and the tests succeed - I will attach a complete patch and review the changes
        Hide
        Henryk Konsek added a comment -

        And how was the review?

        Show
        Henryk Konsek added a comment - And how was the review?
        Hide
        Siegfried Goeschl added a comment -

        Hi Henryk,

        review was as usual - got distracted but had a look at it now

        +) why are you not using LogOutputStream or a similar class? It pretty much does what you want ...
        +) do you have tests/samples how you are using BufferedReaderPumper - I tried to write a test but StreamPumper is only used internally

        Show
        Siegfried Goeschl added a comment - Hi Henryk, review was as usual - got distracted but had a look at it now +) why are you not using LogOutputStream or a similar class? It pretty much does what you want ... +) do you have tests/samples how you are using BufferedReaderPumper - I tried to write a test but StreamPumper is only used internally

          People

          • Assignee:
            Siegfried Goeschl
            Reporter:
            Henryk Konsek
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development