Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15362

FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FileWithSnapshotFeature#updateQuotaAndCollectBlocks uses list to collect blocks 

       List<BlockInfo> allBlocks = new ArrayList<BlockInfo>();
       if (file.getBlocks() != null) {
          allBlocks.addAll(Arrays.asList(file.getBlocks()));
        }

       INodeFile#storagespaceConsumedContiguous collects all distinct blocks by set

      // Collect all distinct blocks
       Set<BlockInfo> allBlocks = new HashSet<>(Arrays.asList(getBlocks()));
       DiffList<FileDiff> diffs = sf.getDiffs().asList();
       for(FileDiff diff : diffs) {
         BlockInfo[] diffBlocks = diff.getBlocks();
         if (diffBlocks != null) {
           allBlocks.addAll(Arrays.asList(diffBlocks));
       } 

      but on updating the reclaim context we subtract these both , so wrong quota value can be updated

      QuotaCounts current = file.storagespaceConsumed(bsp);
      reclaimContext.quotaDelta().add(oldCounts.subtract(current)); 

        Attachments

        1. HDFS-15362.002.patch
          6 kB
          Hemanth Boyina
        2. HDFS-15362.001.patch
          4 kB
          Hemanth Boyina

          Activity

            People

            • Assignee:
              hemanthboyina Hemanth Boyina
              Reporter:
              hemanthboyina Hemanth Boyina
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: