Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8149

cap space usage of default log4j rolling policy

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0-alpha
    • conf
    • None
    • Incompatible change, Reviewed
    • Hide
      Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.
      Show
      Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.

    Description

      I've seen several critical production issues because logs are not automatically removed after some time and accumulate. Changes to Hadoop's default log4j file appender would help with this.

      I recommend we move to an appender which:

      1) caps the max file size (configurable)
      2) caps the max number of files to keep (configurable)
      3) uses rolling file appender rather than DRFA, see the warning here:
      http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
      Specifically: "DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss."

      We'd lose (based on the default log4j configuration) the daily rolling aspect, however increase reliability.

      Attachments

        1. HADOOP-8149.patch
          17 kB
          Patrick D. Hunt
        2. HADOOP-8149.patch
          17 kB
          Patrick D. Hunt
        3. HADOOP-8149.patch
          18 kB
          Patrick D. Hunt
        4. HADOOP-8149.patch
          19 kB
          Patrick D. Hunt
        5. HADOOP-8149.patch
          21 kB
          Patrick D. Hunt

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8216 address log4j.properties inconsistencies btw main and template dirs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-5655 Cap space usage of default log4j rolling policy

          • Major - Major loss of function.
          • Closed
          is required by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-3066 cap space usage of default log4j rolling policy (hdfs specific changes)

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-3989 cap space usage of default log4j rolling policy (mr specific changes)

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. ZOOKEEPER-1435 cap space usage of default log4j rolling policy

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8224 Don't hardcode hdfs.audit.logger in the scripts

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. FLUME-1073 Default Log4j configuration file should have a rolling policy

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              phunt Patrick D. Hunt
              phunt Patrick D. Hunt
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: