Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9839

Reduce verbosity of processReport logging

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      BlockManager#processReport logs one line for each invalidated block at INFO. HDFS-7503 moved this logging outside the NameSystem write lock but we still see the NameNode being slowed down when the number of block invalidations is very large e.g. just after a large amount of data is deleted.

            for (Block b : invalidatedBlocks) {
              blockLog.info("BLOCK* processReport: {} on node {} size {} does not " +
                  "belong to any file", b, node, b.getNumBytes());
            }
      

      We can change this statement to DEBUG and just log the number of block invalidations at INFO.

        Attachments

          Activity

            People

            • Assignee:
              arpitagarwal Arpit Agarwal
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: