Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10434

Is it possible to use "df" to calculate the dfs usage instead of "du"

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 2.3.0
    • None
    • fs

    Description

      When we run datanode from the machine with big disk volume, it's found du operations from org.apache.hadoop.fs.DU's DURefreshThread cost lots of disk performance.

      As we use the whole disk for hdfs storage, it is possible calculate volume usage via "df" command. Is it necessary adding the "df" option for usage calculation in hdfs (org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice)?

      Attachments

        1. HADOOP-10434-1.patch
          5 kB
          MaoYuan Xian

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue HADOOP-12974 Create a CachingGetSpaceUsed implementation that uses df

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8791 block ID-based DN storage layout can be very slow for datanode on ext4

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Activity

            People

              Unassigned Unassigned
              kennethxian MaoYuan Xian
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: