Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5238 Improve NN operation throughput
  3. HDFS-5241

Provide alternate queuing audit logger to reduce logging contention

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha, 3.0.0-alpha1
    • 2.3.0
    • namenode
    • None
    • Reviewed

    Description

      The default audit logger has extremely poor performance. The internal synchronization of log4j causes massive contention between the call handlers (100 by default) which drastically limits the throughput of the NN.

      Attachments

        1. HDFS-5241.patch
          9 kB
          Daryn Sharp
        2. HDFS-5241.patch
          5 kB
          Daryn Sharp

        Issue Links

          is required by

          Sub-task - The sub-task of the issue HDFS-5240 Separate formatting from logging in the audit logger API

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-5823 Document async audit logging

          • Major - Major loss of function.
          • Open

          Activity

            People

              daryn Daryn Sharp
              daryn Daryn Sharp
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: