Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6860

BlockStateChange logs are too noisy

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Block State Change logs are too noisy at the default INFO level and affect NN performance on busy clusters.

      Most of these state changes can be logged at debug level instead.

      Attachments

        1. HDFS6860.patch
          11 kB
          Chang Li
        2. HDFS6860.patch
          13 kB
          Chang Li
        3. HDFS-6860.00.patch
          14 kB
          Xiaoyu Yao
        4. HDFS-6860.01.patch
          12 kB
          Xiaoyu Yao
        5. HDFS-6860.branch-2.6.patch
          97 kB
          Xiaoyu Yao

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14981 BlockStateChange logging is exceedingly verbose

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11430 Add GenericTestUtils#disableLog, GenericTestUtils#setLogLevel

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7706 Switch BlockManager logging to use slf4j

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7712 Switch blockStateChangeLog to use slf4j

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9618 Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9623 Update example configuration of block state change log in log4j.properties

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              lichangleo Chang Li
              arp Arpit Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: