[FLUME-1661] ExecSource cannot execute complex *nix commands

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.4.0
• Component/s: Sinks+Sources
• Release Note: Added new optional config directive 'shell' for Exec source. This enables support for commands that use wildcards, backticks, pipes and other such shell features.
• Assignee: Roshan Naik
• Reporter: Yoonseok Woo
• Created: 19/Oct/12 06:37
• Updated: 11/Mar/13 22:30
• Resolved: 11/Mar/13 19:18

Description

• command line parsing
  • conf/flume.conf

    agent.sources.source1.type = exec
    agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.`date +%Y%m%d%H`
    

  • result

    tail: /some/path/logs/exception/error.log.`date: No such file or directory
    tail: +%Y%m%d%H`: No such file or directory
    

  • needs to be improved

    (ExecSouce.java:242) String[] commandArgs = command.split("\\s+") 
    

• using special character (e.g. *, `, ', ...)
  • conf/flume.conf

    agent.sources.source1.type = exec
    agent.sources.source1.command = tail -f /some/path/logs/exception/error.log.*
    

  • result

    tail: /some/path/logs/exception/error.log.*: No such file or directory
    

  • needs to be improved

    (ExecSouce.java:243) process = new ProcessBuilder(commandArgs).start();
    

Issue Links

• links to: Review Board