Flume
  1. Flume
  2. FLUME-1661

ExecSource cannot execute complex *nix commands

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Release Note:
      Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.

      Description

      • command line parsing
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.`date +%Y%m%d%H`
          
        • result
          tail: /some/path/logs/exception/error.log.`date: No such file or directory
          tail: +%Y%m%d%H`: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:242) String[] commandArgs = command.split("\\s+") 
          
      • using special character (e.g. *, `, ', ...)
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
          
        • result
          tail: /some/path/logs/exception/error.log.*: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
          
      1. FLUME-1661-1.patch
        0.7 kB
        Daisuke Kobayashi
      2. FLUME-1661.patch
        12 kB
        Roshan Naik
      3. FLUME-1661.patch.v2
        12 kB
        Roshan Naik
      4. FLUME-1661.patch.v3
        14 kB
        Roshan Naik
      5. FlumeProcessRunner.tar.gz
        3 kB
        Nitin Verma
      6. FLUME-1661.patch.v4
        14 kB
        Roshan Naik
      7. 1661.patch.v5
        14 kB
        Roshan Naik
      8. 1661.patch.v6
        14 kB
        Roshan Naik
      9. 1661.patch.v7
        15 kB
        Roshan Naik
      10. 1661.patch.v8
        15 kB
        Roshan Naik
      11. 1661.patch.v9
        15 kB
        Roshan Naik

        Issue Links

          Activity

            People

            • Assignee:
              Roshan Naik
              Reporter:
              Yoonseok Woo
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development