Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5655

Cap space usage of default log4j rolling policy

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.1
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This changes the default log rolling scheme from DRFA to RFA. The former rolls over the log on a date change trigger, while the later rolls over when the log file size reaches a predefined limit. The issue with DRFA is that it doesn't have the ability to cap the space usage, so users who are not using host-level log rotation might fill up their log partitions. This results in a cluster crash. RFA puts a size limit on the log size and therefore is a safer option in such scenarios. The default file size is 256MB with 20 files (total of 5GB logs). In case one needs to revert to the original DRFA (for some legacy tools etc), one can set environment variable HBASE_ROOT_LOGGER to <ROOT_LOGGER_LEVEL,DRFA>. Please refer to the hbase-env.sh for more details.
      Show
      This changes the default log rolling scheme from DRFA to RFA. The former rolls over the log on a date change trigger, while the later rolls over when the log file size reaches a predefined limit. The issue with DRFA is that it doesn't have the ability to cap the space usage, so users who are not using host-level log rotation might fill up their log partitions. This results in a cluster crash. RFA puts a size limit on the log size and therefore is a safer option in such scenarios. The default file size is 256MB with 20 files (total of 5GB logs). In case one needs to revert to the original DRFA (for some legacy tools etc), one can set environment variable HBASE_ROOT_LOGGER to <ROOT_LOGGER_LEVEL,DRFA>. Please refer to the hbase-env.sh for more details.
    • Tags:
      dfra, rfa

      Description

      The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead.
      This is in parallel with jira Hadoop-8149.

        Attachments

        1. 5655-v1.patch
          2 kB
          Himanshu Vashishtha
        2. HBase-5655-v2.patch
          2 kB
          Himanshu Vashishtha
        3. HBase-5655-v3.patch
          2 kB
          Himanshu Vashishtha

          Issue Links

            Activity

              People

              • Assignee:
                v.himanshu Himanshu Vashishtha
                Reporter:
                v.himanshu Himanshu Vashishtha
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: