[FLINK-35287] Builder builds NetworkConfig for Elasticsearch connector 8

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: elasticsearch-3.1.0
• Component/s: Connectors / ElasticSearch
• Labels: pull-request-available
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 03/May/24 07:34
• Updated: 22/May/24 04:16
• Resolved: 22/May/24 04:16

Description

In FLINK-26088 we added support for ElasticSearch 8.0. It is based on Async sink API and does not use the base module flink-connector-elasticsearch-base. Regarding the config options (host, username, password, headers, ssl...), we pass all options from the builder to AsyncSink, and last to AsyncWriter. It is less flexible when we add new options and the constructors will get longer and multiple places may validate options unnecessarily. I think it's nice if we make the sink builder builds the NetworkConfig once, and pass it all the way to the writer. This is also how the base module for 6.x / 7.x is implemented. In my recent work adding new options to the network config, this way works simpler.

Let me create a PR to demonstrate the idea. No new features or major code refactoring other than the builder builds the NetworkConfig (code will be shorter). I have a few small fixes which I'll include into the incoming PR.

Issue Links

• links to: GitHub Pull Request #100