Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2657

Enhancements to DFSClient to support flushing data at any point in time

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.17.0
    • None
    • None
    • A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.

    Description

      The HDFS Append Design (HADOOP-1700) requires that there be a public API to flush data written to a HDFS file that can be invoked by an application. This API (popularly referred to a fflush(OutputStream)) will ensure that data written to the DFSOutputStream is flushed to datanodes and any required metadata is persisted on Namenode.

      This API has to handle the case when the client decides to flush after writing data that is not a exact multiple of io.bytes.per.checksum.

      Attachments

        1. flush.patch
          23 kB
          Dhruba Borthakur
        2. flush2.patch
          21 kB
          Dhruba Borthakur
        3. flush3.patch
          22 kB
          Dhruba Borthakur
        4. flush4.patch
          21 kB
          Dhruba Borthakur
        5. flush5.patch
          23 kB
          Dhruba Borthakur
        6. flush6.patch
          23 kB
          Dhruba Borthakur
        7. flush7.patch
          23 kB
          Dhruba Borthakur
        8. flush8.patch
          24 kB
          Dhruba Borthakur
        9. flush9.patch
          25 kB
          Dhruba Borthakur

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dhruba Dhruba Borthakur Assign to me
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment