Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15362

FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.1, 3.4.0
    • 3.3.1, 3.4.0
    • snapshots
    • None
    • Reviewed

    Description

      FileWithSnapshotFeature#updateQuotaAndCollectBlocks uses list to collect blocks 

       List<BlockInfo> allBlocks = new ArrayList<BlockInfo>();
       if (file.getBlocks() != null) {
          allBlocks.addAll(Arrays.asList(file.getBlocks()));
        }

       INodeFile#storagespaceConsumedContiguous collects all distinct blocks by set

      // Collect all distinct blocks
       Set<BlockInfo> allBlocks = new HashSet<>(Arrays.asList(getBlocks()));
       DiffList<FileDiff> diffs = sf.getDiffs().asList();
       for(FileDiff diff : diffs) {
         BlockInfo[] diffBlocks = diff.getBlocks();
         if (diffBlocks != null) {
           allBlocks.addAll(Arrays.asList(diffBlocks));
       } 

      but on updating the reclaim context we subtract these both , so wrong quota value can be updated

      QuotaCounts current = file.storagespaceConsumed(bsp);
      reclaimContext.quotaDelta().add(oldCounts.subtract(current)); 

      Attachments

        1. HDFS-15362.001.patch
          4 kB
          Hemanth Boyina
        2. HDFS-15362.002.patch
          6 kB
          Hemanth Boyina

        Activity

          People

            hemanthboyina Hemanth Boyina
            hemanthboyina Hemanth Boyina
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: