Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.22.0
-
None
-
None
-
IBR, Block Reports, benchmark
Description
Rework NNThroughputBenchmark to provide more detailed info about Initial Block Report (IBR) processing time, as a function of number of nodes, number of unique blocks, and total number of replicas.
Allow both direct local communication and remote and local RPC, so we can see how much impact RPC overhead has on IBR processing time.
Also plug some holes in performance-specific logging of Block Report processing, so that consistent and complete data are logged from both Namenode and Datanode.