Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: hdfs
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      There are some cases of space quota that need to be tested for heterogenous storages. It's quite important to ensure space quota works well in modern clusters typically built out of diversified storage structures. This proposes adding new tests to cover the following scenarios.

      1. Tests space quota for storage policy = WARM
      2. Tests if changing replication factor results in copying file as quota doesn't exceed
      3. Tests if clear quota per heterogenous storage doesnt result in clearing quota for another storage

      The following cases have been covered.

      1. Tests space quota for storage policy = ALL_SSD, covered by TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateAllSSD
      2. Tests if overall space quota exceeds even if particular storage space quota is available, covered by
        TestQuotaByStorageType#testQuotaByStorageTypeAndTraditionalQuotaException3
      3. Tests spaceQuota for storage policy = Cold, covered by TestQuotaByStorageType#testQuotaByStorageTypeWithTraditionalQuota
      4. Tests if quota exceeds for DISK storage even if overall space quota is available, covered by
        TestQuotaByStorageType#testQuotaByStorageTypeAndTraditionalQuotaException2
      5. Tests space quota with append operation, covered by TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateAppend
      6. Sanity Test : Checks if copy command fails if quota is exceeded, covered by
        TestQuotaByStorageType#testQuotaByStorageTypeParent*Child*
      7. Tests space quota with remove operation, covered by TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateDelete
      8. Tests space quota with Snapshot operation, covered by TestQuotaByStorageType#testQuotaByStorageTypeWithSnapshot
      9. Tests space quota with truncate operation, coveved by TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateTruncate
      10. Tests space quota remains valid even with Namenode restart, covered by TestQuotaByStorageType#testQuotaByStorageTypePersistenceInEditLog and TestQuotaByStorageType#testQuotaByStorageTypePersistenceInFsImage

        Attachments

        1. HDFS-11065.000.patch
          7 kB
          Xiaobing Zhou
        2. HDFS-11065.001.patch
          7 kB
          Xiaobing Zhou

          Activity

            People

            • Assignee:
              xiaobingo Xiaobing Zhou
              Reporter:
              xiaobingo Xiaobing Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: