-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
-
Component/s: datanode
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
When a DN has received a block, it immediately sends a block receipt IBR RPC to NN for reporting the block. Even if a DN has received multiple blocks at the same time, it still sends multiple RPCs. It does not scale well since NN has to process a huge number of RPCs when many DNs receiving many blocks at the same time.
- incorporates
-
HDFS-9729 Use null to replace DataNode.EMPTY_DEL_HINT
-
- Resolved
-
- is related to
-
HDFS-10810 Setreplication removing block from underconstrcution temporarily when batch IBR is enabled.
-
- Resolved
-
- relates to
-
HDFS-8999 Allow a file to be closed with COMMITTED but not yet COMPLETE blocks.
-
- Resolved
-
-
HDFS-11837 Backport HDFS-9710 to branch-2.7: Change DN to send block receipt IBRs in batches
-
- Resolved
-
-
HDFS-9726 Refactor IBR code to a new class
-
- Resolved
-