Flume
  1. Flume
  2. FLUME-1661

ExecSource cannot execute complex *nix commands

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Release Note:
      Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.

      Description

      • command line parsing
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.`date +%Y%m%d%H`
          
        • result
          tail: /some/path/logs/exception/error.log.`date: No such file or directory
          tail: +%Y%m%d%H`: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:242) String[] commandArgs = command.split("\\s+") 
          
      • using special character (e.g. *, `, ', ...)
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
          
        • result
          tail: /some/path/logs/exception/error.log.*: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
          
      1. FLUME-1661-1.patch
        0.7 kB
        Daisuke Kobayashi
      2. FLUME-1661.patch
        12 kB
        Roshan Naik
      3. FLUME-1661.patch.v2
        12 kB
        Roshan Naik
      4. FLUME-1661.patch.v3
        14 kB
        Roshan Naik
      5. FlumeProcessRunner.tar.gz
        3 kB
        Nitin Verma
      6. FLUME-1661.patch.v4
        14 kB
        Roshan Naik
      7. 1661.patch.v5
        14 kB
        Roshan Naik
      8. 1661.patch.v6
        14 kB
        Roshan Naik
      9. 1661.patch.v7
        15 kB
        Roshan Naik
      10. 1661.patch.v8
        15 kB
        Roshan Naik
      11. 1661.patch.v9
        15 kB
        Roshan Naik

        Issue Links

          Activity

          Yoonseok Woo created issue -
          Daisuke Kobayashi made changes -
          Field Original Value New Value
          Attachment FLUME-1661-1.patch [ 12550168 ]
          Roshan Naik made changes -
          Fix Version/s v1.4.0 [ 12323372 ]
          Roshan Naik made changes -
          Assignee Roshan Naik [ roshan_naik ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch [ 12550935 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch [ 12550935 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch [ 12550937 ]
          Roshan Naik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.
          Fix Version/s v1.3.0 [ 12322140 ]
          Fix Version/s v1.4.0 [ 12323372 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch.v2 [ 12551269 ]
          Mike Percy made changes -
          Remote Link This issue links to "Review Board (Web Link)" [ 11413 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch.v3 [ 12551964 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch.v3 [ 12551964 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch.v3 [ 12551968 ]
          Nitin Verma made changes -
          Attachment FlumeProcessRunner.tar.gz [ 12551990 ]
          Roshan Naik made changes -
          Attachment FLUME-1661.patch.v4 [ 12552188 ]
          Brock Noland made changes -
          Fix Version/s v1.4.0 [ 12323372 ]
          Fix Version/s v1.3.0 [ 12322140 ]
          Roshan Naik made changes -
          Attachment 1661.patch.v5 [ 12572280 ]
          Roshan Naik made changes -
          Attachment 1661.patch.v6 [ 12572399 ]
          Roshan Naik made changes -
          Attachment 1661.patch.v7 [ 12572582 ]
          Roshan Naik made changes -
          Attachment 1661.patch.v8 [ 12572677 ]
          Roshan Naik made changes -
          Attachment 1661.patch.v9 [ 12572794 ]
          Brock Noland made changes -
          Summary ExecSource cannot execute (little complicated..) *nix commands ExecSource cannot execute complex *nix commands
          Brock Noland made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Roshan Naik
              Reporter:
              Yoonseok Woo
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development