[HDFS-6801] Archival Storage: Add a new data migration tool

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: balancer & mover, namenode
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 01/Aug/14 02:30
• Updated: 19/Sep/14 13:52
• Resolved: 18/Aug/14 17:52

Description

The tool is similar to Balancer. It periodic scans the blocks in HDFS and uses path and/or other meta data (e.g. mtime) to determine if a block should be cooled down (i.e. hot => warm, or warm => cold) or warmed up (i.e. cold => warm, or warm => hot). In contrast to Balancer, the migration tool always move replicas to a different storage type. Similar to Balancer, the replicas are moved in a way that the number of racks the block does not decrease.

Issue Links

• requires: HDFS-6861 Separate Balancer specific logic form Dispatcher
• requires: HDFS-6828 Separate block replica dispatching from Balancer
• requires: HDFS-6809 Move some Balancer's inner classes to standalone classes