[OOZIE-2504] Create a log4j.properties under HADOOP_CONF_DIR in Shell Action

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: 4.3.0
• Component/s: action
• Assignee: Harsh J
• Reporter: Harsh J
• Created: 06/Apr/16 10:38
• Updated: 02/Dec/16 21:04
• Resolved: 14/Jun/16 05:14

Description

We currently do environment exports of HADOOP_CONF_DIR and YARN_CONF_DIR, when the feature of config directory is enabled, for catering to people wanting to use hadoop or such relying commands, and this works well.

However, an issue occurs when <capture-output/> is enabled, wherein the commands, not finding a logger configuration, start using System.out by default [1], and any/all lines that match the capture output criteria from the log-spam in System.out make it into the captured results, which can break further logic done in subsequent actions relying on proper captured values.

We can fix this by pre-preparing, optionally, a log4j.properties file under HADOOP_CONF_DIR and YARN_CONF_DIR, such that the commands use that logging configuration and write logs into System.err by default.

[1] - https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/ConsoleAppender.html

Issue Links

• is broken by: OOZIE-2739 Remove property expansion pattern from ShellMain's log4j properties content