Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14790 Implement a new DFSOutputStream for logging WAL only
  3. HBASE-15709

Handle large edits for asynchronous WAL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • io, wal
    • None
    • Reviewed

    Description

      First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is larger than PacketReceiver.MAX_PACKET_SIZE(16MB).

      Second, since we only allow one block here, we need to make sure we do not exceed the block size after writing a large chunk of data.

      Attachments

        1. HBASE-15709.patch
          13 kB
          Duo Zhang
        2. HBASE-15709-v1.patch
          13 kB
          Duo Zhang
        3. HBASE-15709-v2.patch
          14 kB
          Duo Zhang

        Issue Links

          blocks

          Sub-task - The sub-task of the issue HBASE-15536 Make AsyncFSWAL as our default WAL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: