Hadoop Common
  1. Hadoop Common
  2. HADOOP-2657

Enhancements to DFSClient to support flushing data at any point in time

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.

      Description

      The HDFS Append Design (HADOOP-1700) requires that there be a public API to flush data written to a HDFS file that can be invoked by an application. This API (popularly referred to a fflush(OutputStream)) will ensure that data written to the DFSOutputStream is flushed to datanodes and any required metadata is persisted on Namenode.

      This API has to handle the case when the client decides to flush after writing data that is not a exact multiple of io.bytes.per.checksum.

      1. flush.patch
        23 kB
        dhruba borthakur
      2. flush2.patch
        21 kB
        dhruba borthakur
      3. flush3.patch
        22 kB
        dhruba borthakur
      4. flush4.patch
        21 kB
        dhruba borthakur
      5. flush5.patch
        23 kB
        dhruba borthakur
      6. flush6.patch
        23 kB
        dhruba borthakur
      7. flush7.patch
        23 kB
        dhruba borthakur
      8. flush8.patch
        24 kB
        dhruba borthakur
      9. flush9.patch
        25 kB
        dhruba borthakur

        Issue Links

          Activity

          dhruba borthakur created issue -
          dhruba borthakur made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-1700 [ HADOOP-1700 ]
          dhruba borthakur made changes -
          Attachment flush.patch [ 12376072 ]
          dhruba borthakur made changes -
          Assignee dhruba borthakur [ dhruba ]
          dhruba borthakur made changes -
          Attachment flush2.patch [ 12376536 ]
          dhruba borthakur made changes -
          Attachment flush3.patch [ 12376610 ]
          dhruba borthakur made changes -
          Link This issue relates to HADOOP-2913 [ HADOOP-2913 ]
          dhruba borthakur made changes -
          Attachment flush4.patch [ 12376791 ]
          dhruba borthakur made changes -
          Attachment flush5.patch [ 12377299 ]
          dhruba borthakur made changes -
          Attachment flush6.patch [ 12377392 ]
          dhruba borthakur made changes -
          Fix Version/s 0.17.0 [ 12312913 ]
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          dhruba borthakur made changes -
          Attachment flush7.patch [ 12377504 ]
          dhruba borthakur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          dhruba borthakur made changes -
          Attachment flush8.patch [ 12377511 ]
          dhruba borthakur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Attachment flush9.patch [ 12377553 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          dhruba borthakur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          dhruba borthakur made changes -
          Release Note A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development