Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2657

Enhancements to DFSClient to support flushing data at any point in time

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.

      Description

      The HDFS Append Design (HADOOP-1700) requires that there be a public API to flush data written to a HDFS file that can be invoked by an application. This API (popularly referred to a fflush(OutputStream)) will ensure that data written to the DFSOutputStream is flushed to datanodes and any required metadata is persisted on Namenode.

      This API has to handle the case when the client decides to flush after writing data that is not a exact multiple of io.bytes.per.checksum.

        Attachments

        1. flush.patch
          23 kB
          dhruba borthakur
        2. flush2.patch
          21 kB
          dhruba borthakur
        3. flush3.patch
          22 kB
          dhruba borthakur
        4. flush4.patch
          21 kB
          dhruba borthakur
        5. flush5.patch
          23 kB
          dhruba borthakur
        6. flush6.patch
          23 kB
          dhruba borthakur
        7. flush7.patch
          23 kB
          dhruba borthakur
        8. flush8.patch
          24 kB
          dhruba borthakur
        9. flush9.patch
          25 kB
          dhruba borthakur

          Issue Links

            Activity

              People

              • Assignee:
                dhruba dhruba borthakur
                Reporter:
                dhruba dhruba borthakur
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: