[HDFS-7197] Enhancements to Mover

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: balancer & mover
• Assignee: Benoy Antony
• Reporter: Benoy Antony
• Created: 06/Oct/14 19:15
• Updated: 08/Nov/18 03:37

Description

Data migration tool (mover) was added as part of HDFS-6584.

We have been using Archival storage tier in our clusters. We have implemented a similar data migration tool (Mover) to migrate data to and from Archival storage. This is an umbrella jira to contribute the features and improvements identified based on our experience.

Issue Links

• depends upon: HDFS-6584 Support Archival Storage

Sub-Tasks

• Allow data migration tool to run as a daemon: Resolved
• Reduce impact of laggards on Mover: In Progress
• Allow different values for dfs.datanode.balance.max.concurrent.moves per datanode: In Progress
• Add start and stop wrapper scripts for mover: Open
• Provide storage tier information for a directory via fsck: Closed
• Provide JMX metrics per storage type: Resolved
• Display information per tier on the Namenode UI: Resolved
• Mover should exit with NO_MOVE_BLOCK if no block can be moved: Resolved
• Mover should exit with NO_MOVE_PROGRESS if there is no move progress: Resolved
• Mover can give first priority to local DN if it has target storage type available in local DN: Closed