Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-909

dfs "du" shows that the size of a subdirectory is 0

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.11.0
    • Component/s: None
    • Labels:
      None

      Description

      dfs "du" is implemented by sending a listPaths request to the namenode to get the size of each file/subdir under the directory. At the namenode side, the size of subdir was calculated by recursively going through the whole subtree with the subdir as the root. But starting from the release 0.10.0, the size of subdir is no longer gets calculated. So dfs "du" shows its size as 0.

      The problem is that both "du" and "list" send the same request "listPaths" to the namenode. The previous implmentation made list very expensive, but the current implementation makes du not working.

        Attachments

        1. du.patch
          3 kB
          Hairong Kuang

          Issue Links

            Activity

              People

              • Assignee:
                hairong Hairong Kuang
                Reporter:
                hairong Hairong Kuang
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: