Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
BlockManager#processMisReplicatesAsync is the daemon thread running inside namenode to handle miserplicated blocks. As shown below, it has a trace log for each of the block in the cluster being processed (10000 blocks per iteration after sleep 10s).
MisReplicationResult res = processMisReplicatedBlock(block); if (LOG.isTraceEnabled()) { LOG.trace("block " + block + ": " + res); }
However, it is not very useful as dumping every block in the cluster will overwhelm the namenode log without much useful information assuming the majority of the blocks are not over/under replicated. This ticket is opened to improve the log for easy troubleshooting of block replication related issues by:
1) add debug log for blocks that get under/over replicated result during processMisReplicatedBlock()
2) or change to trace log for only blocks that get non-OK result during processMisReplicatedBlock()