Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2549

hdfs does not honor dfs.du.reserved setting

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.14.4
    • 0.16.0
    • None
    • None
    • FC Linux.

    Description

      running 0.14.4. one of our drives is smaller and is always getting disk full. i reset the disk reservation to 1Gig - but it was filled quickly again.

      i put in some tracing in getnextvolume. the blocksize argument is 0. so every volume (regardless of available space) qualifies. here's the trace:

      /* root disk chosen with 0 available bytes. format is <available>:<blocksize>*/
      2008-01-08 08:08:51,918 WARN org.apache.hadoop.dfs.DataNode: Volume /var/hadoop/tmp/dfs/data/current:0:0

      /* some other disk chosen with 300G space. */
      2008-01-08 08:09:21,974 WARN org.apache.hadoop.dfs.DataNode: Volume /mnt/d1/hdfs/current:304725631026:0

      i am going to default blocksize to something reasonable when it's zero for now.

      this is driving us nuts since our automounter starts failing when we run out of space. so everything's broke.

      Attachments

        1. diskfull.patch
          3 kB
          Hairong Kuang
        2. diskfull1.patch
          4 kB
          Hairong Kuang
        3. diskfull2.patch
          4 kB
          Hairong Kuang

        Issue Links

          Activity

            People

              hairong Hairong Kuang
              jsensarma Joydeep Sen Sarma
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: