Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1661

ExecSource cannot execute complex *nix commands

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.4.0
    • Sinks+Sources
    • None
    • Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.

    Description

      • command line parsing
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.`date +%Y%m%d%H`
          
        • result
          tail: /some/path/logs/exception/error.log.`date: No such file or directory
          tail: +%Y%m%d%H`: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:242) String[] commandArgs = command.split("\\s+") 
          
      • using special character (e.g. *, `, ', ...)
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
          
        • result
          tail: /some/path/logs/exception/error.log.*: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
          

      Attachments

        1. FLUME-1661-1.patch
          0.7 kB
          Daisuke Kobayashi
        2. FLUME-1661.patch
          12 kB
          Roshan Naik
        3. FLUME-1661.patch.v2
          12 kB
          Roshan Naik
        4. FLUME-1661.patch.v3
          14 kB
          Roshan Naik
        5. FlumeProcessRunner.tar.gz
          3 kB
          Nitin Verma
        6. FLUME-1661.patch.v4
          14 kB
          Roshan Naik
        7. 1661.patch.v5
          14 kB
          Roshan Naik
        8. 1661.patch.v6
          14 kB
          Roshan Naik
        9. 1661.patch.v7
          15 kB
          Roshan Naik
        10. 1661.patch.v8
          15 kB
          Roshan Naik
        11. 1661.patch.v9
          15 kB
          Roshan Naik

        Issue Links

          Activity

            People

              roshan_naik Roshan Naik
              urecho Yoonseok Woo
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: