Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-895

Allow hflush/sync to occur in parallel with new writes to the file

    Details

    • Hadoop Flags:
      Reviewed

      Description

      In the current trunk, the HDFS client methods writeChunk() and hflush./sync are syncronized. This means that if a hflush/sync is in progress, an applicationn cannot write data to the HDFS client buffer. This reduces the write throughput of the transaction log in HBase.

      The hflush/sync should allow new writes to happen to the HDFS client even when a hflush/sync is in progress. It can record the seqno of the message for which it should receice the ack, indicate to the DataStream thread to star flushing those messages, exit the synchronized section and just wai for that ack to arrive.

      1. 895-delta-for-review.txt
        7 kB
        Todd Lipcon
      2. ASF.LICENSE.NOT.GRANTED--hdfs-895.txt
        15 kB
        Todd Lipcon
      3. ASF.LICENSE.NOT.GRANTED--hdfs-895-trunk.txt
        19 kB
        Todd Lipcon
      4. HDFS-895.20-security.1.patch
        18 kB
        Jitendra Nath Pandey
      5. hdfs-895.txt
        22 kB
        Todd Lipcon
      6. hdfs-895.txt
        21 kB
        Todd Lipcon
      7. hdfs-895.txt
        22 kB
        Todd Lipcon
      8. hdfs-895.txt
        22 kB
        Todd Lipcon
      9. hdfs-895.txt
        21 kB
        Todd Lipcon
      10. hdfs-895.txt
        17 kB
        Todd Lipcon
      11. hdfs-895-0.20-append.txt
        19 kB
        Todd Lipcon
      12. hdfs-895-0.20-append.txt
        16 kB
        Todd Lipcon
      13. hdfs-895-20.txt
        16 kB
        Todd Lipcon
      14. hdfs-895-branch-20-append.txt
        17 kB
        Todd Lipcon
      15. hdfs-895-ontopof-1497.txt
        23 kB
        Todd Lipcon
      16. hdfs-895-review.txt
        20 kB
        Todd Lipcon

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              dhruba borthakur
            • Votes:
              1 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development