[IMPALA-4134] Use Kudu AUTO_FLUSH_BACKGROUND mode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Kudu_Impala
• Fix Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: kudu performance
• Target Version: Kudu_Impala
• Assignee: Matthew Jacobs
• Reporter: Matthew Jacobs
• Created: 14/Sep/16 22:02
• Updated: 25/Oct/16 15:48
• Resolved: 25/Oct/16 15:48

Description

Kudu supports writing with AUTO_FLUSH_BACKGROUND mode as of Kudu 1.0 for flushing buffered write operations:

See https://github.com/apache/kudu/blob/branch-1.0.x/src/kudu/client/client.h#L1157

This may improve performance in some cases, so we should test this and consider switching.

From Alexey Serbin:

I did my testing with simple 'push-as-mush-as-client-can-do' scenarios, and results look good (will share a link to the performance summary soon): the new code performs comparable with the old one if both run in MANUAL_FLUSH mode. And of course, a session in AUTO_FLUSH_BACKGROUND mode perform much better that session in AUTO_FLUSH_SYNC mode. Also, session in AUTO_FLUSH_BACKGROUND mode performs better than session in MANUAL_FLUSH mode if the buffer of the former allows to accommodate more operations than the latter flushes time to time.