Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1661

ExecSource cannot execute complex *nix commands

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Release Note:
      Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.

      Description

      • command line parsing
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.`date +%Y%m%d%H`
          
        • result
          tail: /some/path/logs/exception/error.log.`date: No such file or directory
          tail: +%Y%m%d%H`: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:242) String[] commandArgs = command.split("\\s+") 
          
      • using special character (e.g. *, `, ', ...)
        • conf/flume.conf
          agent.sources.source1.type = exec
          agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
          
        • result
          tail: /some/path/logs/exception/error.log.*: No such file or directory
          
        • needs to be improved
          (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
          

        Attachments

        1. FlumeProcessRunner.tar.gz
          3 kB
          Nitin Verma
        2. FLUME-1661-1.patch
          0.7 kB
          Daisuke Kobayashi
        3. FLUME-1661.patch.v4
          14 kB
          Roshan Naik
        4. FLUME-1661.patch.v3
          14 kB
          Roshan Naik
        5. FLUME-1661.patch.v2
          12 kB
          Roshan Naik
        6. FLUME-1661.patch
          12 kB
          Roshan Naik
        7. 1661.patch.v9
          15 kB
          Roshan Naik
        8. 1661.patch.v8
          15 kB
          Roshan Naik
        9. 1661.patch.v7
          15 kB
          Roshan Naik
        10. 1661.patch.v6
          14 kB
          Roshan Naik
        11. 1661.patch.v5
          14 kB
          Roshan Naik

          Issue Links

            Activity

              People

              • Assignee:
                roshan_naik Roshan Naik
                Reporter:
                urecho Yoonseok Woo
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: