Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4442

Disable the location information of audit logger to reduce overhead

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Implemented
    • 1.0.0
    • 1.1.0
    • OM
    • None

    Description

      Log4j2 has substantial overhead with the default configuration. We found more than 20%~30% OM overhead in a microbenchmark (in which clients sends lookupFile calls)  was spent on audit logging, even though we use async audit logger.

      Note that while the log4j2 documentation states the location information is disabled by default, we found that explicitly disabling location information has visible performance improvement.

      Solution:
      add rootLogger.includeLocation=false to OM's audit logger property (om-audit-log4j2.properties file)

      After this change, the audit logger overhead reduced to 7%

      According to LOG4J2-2644, log4j2 2.12.1 and above improved this. So Ozone 1.1 should be in much better shape. File this Jira for future reference. Maybe it'll benefit other projects that use older Log4j2.

      Credit: adoroszlai for identifying the solution.

       

      Details on log4j2 location information: https://logging.apache.org/log4j/2.x/manual/layouts.html#LocationInformation

      Attachments

        1. Screen Shot 2020-11-09 at 10.54.05 AM.png
          1.50 MB
          Wei-Chiu Chuang
        2. Screen Shot 2020-11-09 at 12.29.34 PM.png
          1.87 MB
          Wei-Chiu Chuang

        Issue Links

          Activity

            People

              adoroszlai Attila Doroszlai
              weichiu Wei-Chiu Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: