Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6898

DN must reserve space for a full block when an RBW block is created

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.5.0
    • 2.6.0
    • datanode
    • None
    • Reviewed

    Description

      DN will successfully create two RBW blocks on the same volume even if the free space is sufficient for just one full block.

      One or both block writers may subsequently get a DiskOutOfSpace exception. This can be avoided by allocating space up front.

      Attachments

        1. HDFS-6898.01.patch
          24 kB
          Arpit Agarwal
        2. HDFS-6898.03.patch
          26 kB
          Arpit Agarwal
        3. HDFS-6898.04.patch
          31 kB
          Arpit Agarwal
        4. HDFS-6898.05.patch
          31 kB
          Arpit Agarwal
        5. HDFS-6898.06.patch
          31 kB
          Arpit Agarwal
        6. HDFS-6898.07.patch
          29 kB
          Arpit Agarwal

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8626 Reserved RBW space is not released if creation of RBW File fails

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9530 ReservedSpace is not cleared for abandoned Blocks

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8072 Reserved RBW space is not released if client terminates while writing block

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue HDFS-7027 Archival Storage: Mover does not terminate when some storage type is out of space

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8547 The space reservation for RBW block is leaking

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is required by

          New Feature - A new feature of the product, which has yet to be developed. HDFS-6581 Write to single replica in memory

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6955 DN should reserve disk space for a full block when creating tmp files

          • Major - Major loss of function.
          • Resolved
          requires

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6758 block writer should pass the expected block size to DataXceiverServer

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6899 Allow changing MiniDFSCluster volumes per DN and capacity per volume

          • Major - Major loss of function.
          • Closed

          Activity

            People

              arp Arpit Agarwal
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: