Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2504

Create a log4j.properties under HADOOP_CONF_DIR in Shell Action

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 4.3.0
    • Component/s: action
    • Labels:
      None

      Description

      We currently do environment exports of HADOOP_CONF_DIR and YARN_CONF_DIR, when the feature of config directory is enabled, for catering to people wanting to use hadoop or such relying commands, and this works well.

      However, an issue occurs when <capture-output/> is enabled, wherein the commands, not finding a logger configuration, start using System.out by default [1], and any/all lines that match the capture output criteria from the log-spam in System.out make it into the captured results, which can break further logic done in subsequent actions relying on proper captured values.

      We can fix this by pre-preparing, optionally, a log4j.properties file under HADOOP_CONF_DIR and YARN_CONF_DIR, such that the commands use that logging configuration and write logs into System.err by default.

      [1] - https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/ConsoleAppender.html

        Attachments

        1. OOZIE-2504.001.patch
          17 kB
          Harsh J
        2. OOZIE-2504.000.patch
          18 kB
          Harsh J

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                qwertymaniac Harsh J
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: