Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2078

Flume sink fails to write if the size in bytes of the batch exceeds the client session's manual flush buffer size

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: flume-sink
    • Labels:
      None

      Description

      The Flume sink uses MANUAL_FLUSH mode. If a set of operations is too large for the flush buffer, it will fail-- potentially over and over, if the channel supports retrying the transaction.

      We should be able to fix this by using AUTO_FLUSH_BACKGROUND, adjusting the buffer size, or returning a more useful error so that users can reconfigure their sink's batch size (though that size is in # of ops, not bytes).

        Attachments

          Activity

            People

            • Assignee:
              wdberkeley William Berkeley
              Reporter:
              wdberkeley William Berkeley
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: