Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1189

Quota counts missed between clear quota and set quota

    Details

    • Hadoop Flags:
      Reviewed
    • Tags:
      hdfs quota

      Description

      HDFS Quota counts will be missed between a clear quota operation and a set quota.

      When setting quota for a dir, the INodeDirectory will be replaced by INodeDirectoryWithQuota and dir.isQuotaSet() becomes true. When INodeDirectoryWithQuota is newly created, quota counting will be performed. However, when clearing quota, the quota conf is set to -1 and dir.isQuotaSet() becomes false while INodeDirectoryWithQuota will NOT be replaced back to INodeDirectory.

      FSDirectory.updateCount just update the quota count for inodes that isQuotaSet() is true. So after clear quota for a dir, its quota counts will not be updated and it's reasonable. But when re seting quota for this dir, quota counting will not be performed and some counts will be missed.

      1. HDFS-1189-for_20.204.patch
        4 kB
        John George
      2. HDFS-1189-for_20.204.patch
        4 kB
        John George
      3. hdfs-1189-1.patch
        4 kB
        John George
      4. HDFS-1189.patch
        4 kB
        John George
      5. HDFS-1189.patch
        4 kB
        John George
      6. HDFS-1189.patch
        1 kB
        Kang Xiao

        Issue Links

          Activity

          Kang Xiao created issue -
          Kang Xiao made changes -
          Field Original Value New Value
          Attachment HDFS-1189.patch [ 12446462 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HDFS-1258 [ HDFS-1258 ]
          John George made changes -
          Attachment HDFS-1189.patch [ 12471990 ]
          John George made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee John George [ johnvijoe ]
          Fix Version/s 0.22.0 [ 12314241 ]
          John George made changes -
          Attachment HDFS-1189.patch [ 12471993 ]
          John George made changes -
          Attachment hdfs-1189-1.patch [ 12472191 ]
          John George made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          John George made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.1 [ 12315271 ]
          Fix Version/s 0.23.0 [ 12315571 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Fix Version/s Federation Branch [ 12315300 ]
          John George made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          John George made changes -
          Attachment HDFS-1189-for_20.204.patch [ 12475100 ]
          John George made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hadoop Flags [Reviewed]
          Affects Version/s 0.20.204.0 [ 12316319 ]
          Affects Version/s 0.20.2 [ 12314204 ]
          Affects Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.20.204.0 [ 12316319 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.21.1 [ 12315271 ]
          Fix Version/s Federation Branch [ 12315300 ]
          Fix Version/s 0.23.0 [ 12315571 ]
          Tsz Wo Nicholas Sze made changes -
          Fix Version/s 0.21.1 [ 12315271 ]
          Fix Version/s Federation Branch [ 12315300 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.23.0 [ 12315571 ]
          Affects Version/s 0.20.204.0 [ 12316319 ]
          John George made changes -
          Attachment HDFS-1189-for_20.204.patch [ 12475258 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Owen O'Malley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              John George
              Reporter:
              Kang Xiao
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development