[FLINK-16497] Improve default flush strategy for JDBC sink to make it work out-of-box

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Connectors / JDBC, Table SQL / Ecosystem
• Labels: pull-request-available
• Assignee: Jark Wu
• Reporter: Jark Wu
• Created: 09/Mar/20 03:42
• Updated: 11/Jun/20 09:05
• Resolved: 11/Jun/20 09:05

Description

Currently, old JDBC sink provides 2 flush options:

'connector.write.flush.max-rows' = '5000', -- default is 5000
'connector.write.flush.interval' = '2s', -- no default value

That means if flush interval is not set, the buffered output rows may not be flushed to database for a long time. That is a surprising behavior because no results are outputed by default.

So we propose to have a default flush '1s' interval and '100' rows for JDBC sink flush. This only applies to new JDBC sink options:

'sink.buffer-flush.max-rows' = '100'
'sink.buffer-flush.interval' = '1s'

Issue Links

• duplicates: FLINK-18041 Make a little bit improvement for DEFAULT_FLUSH_MAX_SIZE and DEFAULT_FLUSH_INTERVAL_MILLS of AbstractJdbcOutputFormat
• relates to: FLINK-16495 Improve default flush strategy for Elasticsearch sink to make it work out-of-box
• relates to: FLINK-16496 Improve default flush strategy for HBase sink to make it work out-of-box
• links to: GitHub Pull Request #12536