Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-10531

Add EC policy and storage policy related usage summarization function to dfs du command

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: None

      Description

      Currently du command output:

              [ ~]$ hdfs dfs -du  -h /home/rgao/
              0      /home/rgao/.Trash
              0      /home/rgao/.staging
              100 M  /home/rgao/ds
              250 M  /home/rgao/ds-2
              200 M  /home/rgao/noECBackup-ds
              500 M  /home/rgao/noECBackup-ds-2
      

      For hdfs users and administrators, EC policy and storage policy related usage summarization would be very helpful when managing storages of cluster. The imitate output of du could be like the following.

              [ ~]$ hdfs dfs -du  -h -t( total, parameter to be added) /home/rgao
               
              0      /home/rgao/.Trash
              0      /home/rgao/.staging
              [Archive] [EC:RS-DEFAULT-6-3-64k] 100 M  /home/rgao/ds
              [DISK] [EC:RS-DEFAULT-6-3-64k]     250 M  /home/rgao/ds-2
              [DISK] [Replica]     200 M  /home/rgao/noECBackup-ds
              [DISK] [Replica]     500 M  /home/rgao/noECBackup-ds-2
               
              Total:
               
              [Archive][EC:RS-DEFAULT-6-3-64k]  100 M
              [Archive][Replica]                                0 M
              [DISK] [EC:RS-DEFAULT-6-3-64k]     250 M
              [DISK] [Replica]                               700 M  
           
              [Archive][ALL]                                 100M
              [DISK]    [ALL]                                  950M
              [ALL]     [EC:RS-DEFAULT-6-3-64k]    350M
              [ALL]     [Replica]                              700M
      

        Attachments

        1. HDFS-10531.001.patch
          19 kB
          Wei-Chiu Chuang

          Activity

            People

            • Assignee:
              Sammi Sammi Chen
              Reporter:
              demongaorui Rui Gao
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: