Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10824

MiniDFSCluster#storageCapacities has no effects on real capacity

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It has been noticed MiniDFSCluster#storageCapacities has no effects on real capacity. It can be reproduced by explicitly setting storageCapacities and then call ClientProtocol#getDatanodeStorageReport(DatanodeReportType.LIVE) to compare results. The following are storage report for one node with two volumes after I set capacity as 300 * 1024. Apparently, the capacity is not changed.

      adminState|DatanodeInfo$AdminStates (id=6861)

      blockPoolUsed 215192
      cacheCapacity 0
      cacheUsed 0
      capacity 998164971520
      datanodeUuid "839912e9-5bcb-45d1-81cf-9a9c9c02a00b" (id=6862)
      dependentHostNames LinkedList<E> (id=6863)
      dfsUsed 215192
      hostName "127.0.0.1" (id=6864)
      infoPort 64222
      infoSecurePort 0
      ipAddr "127.0.0.1" (id=6865)
      ipcPort 64223
      lastUpdate 1472682790948
      lastUpdateMonotonic 209605640
      level 0
      location "/default-rack" (id=6866)
      maintenanceExpireTimeInMS 0
      parent null
      peerHostName null
      remaining 20486512640
      softwareVersion null
      upgradeDomain null
      xceiverCount 1
      xferAddr "127.0.0.1:64220" (id=6855)
      xferPort 64220

      [0]StorageReport (id=6856)

      blockPoolUsed 4096
      capacity 499082485760
      dfsUsed 4096
      failed false
      remaining 10243256320
      storage DatanodeStorage (id=6869)

      [1]StorageReport (id=6859)

      blockPoolUsed 211096
      capacity 499082485760
      dfsUsed 211096
      failed false
      remaining 10243256320
      storage DatanodeStorage (id=6872)

        Attachments

        1. HDFS-10824-branch-2.006.patch
          11 kB
          Xiaobing Zhou
        2. HDFS-10824.006.patch
          11 kB
          Xiaobing Zhou
        3. HDFS-10824.005.patch
          11 kB
          Xiaobing Zhou
        4. HDFS-10824.004.patch
          11 kB
          Xiaobing Zhou
        5. HDFS-10824.003.patch
          10 kB
          Xiaobing Zhou
        6. HDFS-10824.002.patch
          10 kB
          Xiaobing Zhou
        7. HDFS-10824.001.patch
          12 kB
          Xiaobing Zhou
        8. HDFS-10824.000.patch
          11 kB
          Xiaobing Zhou

          Activity

            People

            • Assignee:
              xiaobingo Xiaobing Zhou
              Reporter:
              xiaobingo Xiaobing Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: