[HDFS-10531] Add EC policy and storage policy related usage summarization function to dfs du command

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0-alpha1
• Labels: hdfs-ec-3.0-nice-to-have
• Target Version/s: 3.0.0-beta1
• Assignee: Sammi Chen
• Reporter: Rui Gao
• Created: 15/Jun/16 06:00
• Updated: 18/Aug/17 18:27
• Resolved: 18/Aug/17 18:27

Description

Currently du command output:

        [ ~]$ hdfs dfs -du  -h /home/rgao/
        0      /home/rgao/.Trash
        0      /home/rgao/.staging
        100 M  /home/rgao/ds
        250 M  /home/rgao/ds-2
        200 M  /home/rgao/noECBackup-ds
        500 M  /home/rgao/noECBackup-ds-2

For hdfs users and administrators, EC policy and storage policy related usage summarization would be very helpful when managing storages of cluster. The imitate output of du could be like the following.

        [ ~]$ hdfs dfs -du  -h -t( total, parameter to be added) /home/rgao
         
        0      /home/rgao/.Trash
        0      /home/rgao/.staging
        [Archive] [EC:RS-DEFAULT-6-3-64k] 100 M  /home/rgao/ds
        [DISK] [EC:RS-DEFAULT-6-3-64k]     250 M  /home/rgao/ds-2
        [DISK] [Replica]     200 M  /home/rgao/noECBackup-ds
        [DISK] [Replica]     500 M  /home/rgao/noECBackup-ds-2
         
        Total:
         
        [Archive][EC:RS-DEFAULT-6-3-64k]  100 M
        [Archive][Replica]                                0 M
        [DISK] [EC:RS-DEFAULT-6-3-64k]     250 M
        [DISK] [Replica]                               700 M  
     
        [Archive][ALL]                                 100M
        [DISK]    [ALL]                                  950M
        [ALL]     [EC:RS-DEFAULT-6-3-64k]    350M
        [ALL]     [Replica]                              700M