Status: Patch Available
Affects Version/s: None
Fix Version/s: None
Erasure Coding: Improve memory efficiency of BlockInfoStriped
Assume we have a BlockInfoStriped:
s0 means storage_0
When we run balancer/mover to re-locate replica on s2, firstly it becomes:
Then the replica on s2 is removed, finally it becomes:
The worst case is:
We should learn from BlockInfoContiguous.removeStorage(..). When a storage is removed, we move the last item front.
With the improvement, the worst case become:
We have an empty slot.
Assume we copy 4 storage first, then delete 4. Even with the improvement, the worst case could be:
But the Balancer uses delHint. So when add one will always delete one. So this case won't happen for striped and contiguous blocks.
idx_i must be moved to slot_i. So slot_i will have idx_i. So we can do further improvement in HDFS-8032.