[HDFS-10598] DiskBalancer does not execute multi-steps plan.

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.0.0-beta1
• Fix Version/s: 3.0.0-alpha1
• Component/s: diskbalancer
• Target Version/s: 3.0.0-beta1
• Hadoop Flags: Reviewed
• Assignee: Lei (Eddy) Xu
• Reporter: Lei (Eddy) Xu
• Created: 06/Jul/16 00:07
• Updated: 02/Oct/19 17:13
• Resolved: 26/Aug/16 22:53

Description

I set up a 3 DN node cluster, each one with 2 small disks. After creating some files to fill HDFS, I added two more small disks to one DN. And run the diskbalancer on this DataNode.

The disk usage before running diskbalancer:

/dev/loop0  3.9G  2.1G  1.6G 58%  /mnt/data1
/dev/loop1  3.9G  2.6G  1.1G 71%  /mnt/data2
/dev/loop2  3.9G  17M  3.6G 1%  /mnt/data3
/dev/loop3  3.9G  17M  3.6G 1%  /mnt/data4

However, after running diskbalancer (i.e., -query shows PLAN_DONE)

/dev/loop0  3.9G  1.2G  2.5G 32%  /mnt/data1
/dev/loop1  3.9G  2.6G  1.1G 71%  /mnt/data2
/dev/loop2  3.9G  953M  2.7G 26%  /mnt/data3
/dev/loop3  3.9G  17M  3.6G 1%   /mnt/data4

It is suspicious that in DiskBalancerMover#copyBlocks, every return does this.setExitFlag which prevents copyBlocks() be called multiple times from DiskBalancer#executePlan.

Issue Links

• is related to: HDFS-10808 DiskBalancer does not execute multi-steps plan-redux