Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6898

DN must reserve space for a full block when an RBW block is created

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      DN will successfully create two RBW blocks on the same volume even if the free space is sufficient for just one full block.

      One or both block writers may subsequently get a DiskOutOfSpace exception. This can be avoided by allocating space up front.

        Attachments

        1. HDFS-6898.01.patch
          24 kB
          Arpit Agarwal
        2. HDFS-6898.03.patch
          26 kB
          Arpit Agarwal
        3. HDFS-6898.04.patch
          31 kB
          Arpit Agarwal
        4. HDFS-6898.05.patch
          31 kB
          Arpit Agarwal
        5. HDFS-6898.06.patch
          31 kB
          Arpit Agarwal
        6. HDFS-6898.07.patch
          29 kB
          Arpit Agarwal

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8626 Reserved RBW space is not released if creation of RBW File fails

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9530 ReservedSpace is not cleared for abandoned Blocks

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8072 Reserved RBW space is not released if client terminates while writing block

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue HDFS-7027 Archival Storage: Mover does not terminate when some storage type is out of space

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8547 The space reservation for RBW block is leaking

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is required by

          New Feature - A new feature of the product, which has yet to be developed. HDFS-6581 Write to single replica in memory

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6955 DN should reserve disk space for a full block when creating tmp files

          • Major - Major loss of function.
          • Resolved
          requires

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6758 block writer should pass the expected block size to DataXceiverServer

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6899 Allow changing MiniDFSCluster volumes per DN and capacity per volume

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                arp Arpit Agarwal
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: