Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4134

Use Kudu AUTO_FLUSH_BACKGROUND mode

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Kudu_Impala
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      Kudu supports writing with AUTO_FLUSH_BACKGROUND mode as of Kudu 1.0 for flushing buffered write operations:

      See https://github.com/apache/kudu/blob/branch-1.0.x/src/kudu/client/client.h#L1157

      This may improve performance in some cases, so we should test this and consider switching.

      From Alexey Serbin:

      I did my testing with simple 'push-as-mush-as-client-can-do' scenarios, and results look good (will share a link to the performance summary soon): the new code performs comparable with the old one if both run in MANUAL_FLUSH mode. And of course, a session in AUTO_FLUSH_BACKGROUND mode perform much better that session in AUTO_FLUSH_SYNC mode. Also, session in AUTO_FLUSH_BACKGROUND mode performs better than session in MANUAL_FLUSH mode if the buffer of the former allows to accommodate more operations than the latter flushes time to time.

        Attachments

          Activity

            People

            • Assignee:
              mjacobs Matthew Jacobs
              Reporter:
              mjacobs Matthew Jacobs
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: