Hadoop Common
  1. Hadoop Common
  2. HADOOP-8149

cap space usage of default log4j rolling policy

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha
    • Component/s: conf
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.
      Show
      Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.

      Description

      I've seen several critical production issues because logs are not automatically removed after some time and accumulate. Changes to Hadoop's default log4j file appender would help with this.

      I recommend we move to an appender which:

      1) caps the max file size (configurable)
      2) caps the max number of files to keep (configurable)
      3) uses rolling file appender rather than DRFA, see the warning here:
      http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
      Specifically: "DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss."

      We'd lose (based on the default log4j configuration) the daily rolling aspect, however increase reliability.

      1. HADOOP-8149.patch
        21 kB
        Patrick Hunt
      2. HADOOP-8149.patch
        19 kB
        Patrick Hunt
      3. HADOOP-8149.patch
        18 kB
        Patrick Hunt
      4. HADOOP-8149.patch
        17 kB
        Patrick Hunt
      5. HADOOP-8149.patch
        17 kB
        Patrick Hunt

        Issue Links

          Activity

          Patrick Hunt created issue -
          Patrick Hunt made changes -
          Field Original Value New Value
          Attachment HADOOP-8149.patch [ 12517443 ]
          Patrick Hunt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Patrick Hunt made changes -
          Link This issue is required by HDFS-3066 [ HDFS-3066 ]
          Patrick Hunt made changes -
          Link This issue is required by MAPREDUCE-3989 [ MAPREDUCE-3989 ]
          Patrick Hunt made changes -
          Attachment HADOOP-8149.patch [ 12517633 ]
          Patrick Hunt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Eli Collins made changes -
          Assignee Patrick Hunt [ phunt ]
          Eli Collins made changes -
          Target Version/s 0.23.3 [ 12320059 ]
          Eli Collins made changes -
          Hadoop Flags Incompatible change,Reviewed [ 10342,10343 ]
          Release Note Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Patrick Hunt made changes -
          Attachment HADOOP-8149.patch [ 12518569 ]
          Eli Collins made changes -
          Assignee Patrick Hunt [ phunt ] Eli Collins [ eli2 ]
          Eli Collins made changes -
          Assignee Eli Collins [ eli2 ] Patrick Hunt [ phunt ]
          Eli Collins made changes -
          Link This issue is related to HADOOP-8216 [ HADOOP-8216 ]
          Eli Collins made changes -
          Link This issue is related to HBASE-5655 [ HBASE-5655 ]
          Patrick Hunt made changes -
          Attachment HADOOP-8149.patch [ 12520333 ]
          Patrick Hunt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively. Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER environment variable to use DRFA.
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Patrick Hunt made changes -
          Attachment HADOOP-8149.patch [ 12520335 ]
          Patrick Hunt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER environment variable to use DRFA. Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.
          Eli Collins made changes -
          Link This issue relates to HADOOP-8224 [ HADOOP-8224 ]
          Eli Collins made changes -
          Link This issue relates to FLUME-1073 [ FLUME-1073 ]
          Eli Collins made changes -
          Link This issue relates to ZOOKEEPER-1435 [ ZOOKEEPER-1435 ]
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively. Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.
          Target Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 2.0.0 [ 12320352 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Patrick Hunt
              Reporter:
              Patrick Hunt
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development