[BAHIR-228] Flink SQL supports kudu sink

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: Flink-1.1.0
• Component/s: Flink Streaming Connectors
• Assignee: dalongliu
• Reporter: dalongliu
• Created: 21/Mar/20 07:47
• Updated: 28/Jul/20 18:45
• Resolved: 28/Jul/20 18:43

Description

currently, for Flink-1.10.0, we can use the catalog to store our stream table sink for kudu, it should exist a kudu table sink so we can register it to catalog, and use kudu as a table in SQL environment.

we can use kudu table sink like this:

KuduOptions options = KuduOptions.builder() 
            .setKuduMaster(kuduMaster) 
            .setTableName(kuduTable) 
            .build(); 
KuduWriterOptions writerOptions = KuduWriterOptions.builder()          .setWriteMode(KuduWriterMode.UPSERT) 
            .setFlushMode(FlushMode.AUTO_FLUSH_BACKGROUND) 
            .build(); 
KuduTableSink tableSink = KuduTableSink.builder() 
            .setOptions(options) 
            .setWriterOptions(writerOptions) 
            .setTableSchema(schema) 
            .build(); 
tEnv.registerTableSink("kudu", tableSink);  
tEnv.sqlUpdate("insert into kudu select * from source");

I have used kudu table sink to sync data in company's production environment, the writing  speed at 5w/s in upsert mode