Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7584

Enable Quota Support for Storage Types

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.7.0
    • datanode, namenode
    • None
    • Reviewed
    • Hide
      1. Introduced quota by storage type as a hard limit on the amount of space usage allowed for different storage types (SSD, DISK, ARCHIVE) under the target directory.
      2. Added {{SetQuotaByStorageType}} API and {{-storagetype}} option for {{hdfs dfsadmin -setSpaceQuota/-clrSpaceQuota}} commands to allow set/clear quota by storage type under the target directory.
      Show
      1. Introduced quota by storage type as a hard limit on the amount of space usage allowed for different storage types (SSD, DISK, ARCHIVE) under the target directory. 2. Added {{SetQuotaByStorageType}} API and {{-storagetype}} option for {{hdfs dfsadmin -setSpaceQuota/-clrSpaceQuota}} commands to allow set/clear quota by storage type under the target directory.

    Description

      Phase II of the Heterogeneous storage features have completed by HDFS-6584. This JIRA is opened to enable Quota support of different storage types in terms of storage space usage. This is more important for certain storage types such as SSD as it is precious and more performant.

      As described in the design doc of HDFS-5682, we plan to add new quotaByStorageType command and new name node RPC protocol for it. The quota by storage type feature is applied to HDFS directory level similar to traditional HDFS space quota.

      Attachments

        1. HDFS-7584.9c.patch
          273 kB
          Xiaoyu Yao
        2. HDFS-7584.9b.patch
          273 kB
          Xiaoyu Yao
        3. HDFS-7584.9a.patch
          273 kB
          Xiaoyu Yao
        4. editsStored
          6 kB
          Xiaoyu Yao
        5. HDFS-7584.9.patch
          272 kB
          Xiaoyu Yao
        6. HDFS-7584.8.patch
          277 kB
          Xiaoyu Yao
        7. HDFS-7584.7.patch
          277 kB
          Xiaoyu Yao
        8. HDFS-7584.6.patch
          278 kB
          Xiaoyu Yao
        9. HDFS-7584.5.patch
          247 kB
          Xiaoyu Yao
        10. HDFS-7584.4.patch
          273 kB
          Xiaoyu Yao
        11. HDFS-7584.3.patch
          270 kB
          Xiaoyu Yao
        12. HDFS-7584.2.patch
          264 kB
          Xiaoyu Yao
        13. HDFS-7584.1.patch
          263 kB
          Xiaoyu Yao
        14. HDFS-7584.0.patch
          231 kB
          Xiaoyu Yao
        15. HDFS-7584 Quota by Storage Type - 01202015.pdf
          133 kB
          Xiaoyu Yao

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8711 setSpaceQuota command should print the available storage type when input storage type is wrong

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9234 WebHdfs : getContentSummary() should give quota for storage types

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-7728 Avoid updating quota usage while loading edits

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              xyao Xiaoyu Yao
              xyao Xiaoyu Yao
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: