Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2657

Enhancements to DFSClient to support flushing data at any point in time

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.17.0
    • None
    • None
    • A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.

    Description

      The HDFS Append Design (HADOOP-1700) requires that there be a public API to flush data written to a HDFS file that can be invoked by an application. This API (popularly referred to a fflush(OutputStream)) will ensure that data written to the DFSOutputStream is flushed to datanodes and any required metadata is persisted on Namenode.

      This API has to handle the case when the client decides to flush after writing data that is not a exact multiple of io.bytes.per.checksum.

      Attachments

        1. flush.patch
          23 kB
          Dhruba Borthakur
        2. flush2.patch
          21 kB
          Dhruba Borthakur
        3. flush3.patch
          22 kB
          Dhruba Borthakur
        4. flush4.patch
          21 kB
          Dhruba Borthakur
        5. flush5.patch
          23 kB
          Dhruba Borthakur
        6. flush6.patch
          23 kB
          Dhruba Borthakur
        7. flush7.patch
          23 kB
          Dhruba Borthakur
        8. flush8.patch
          24 kB
          Dhruba Borthakur
        9. flush9.patch
          25 kB
          Dhruba Borthakur

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-1700 Append to files in HDFS

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-24 FSDataOutputStream should flush last partial CRC chunk

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              dhruba Dhruba Borthakur
              dhruba Dhruba Borthakur
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: