Details
-
Improvement
-
Status: Triage Needed
-
P2
-
Resolution: Done
-
None
-
None
Description
Currently, the Apache Beam Elasticsearch sink is using the index bulk API operation to add data to the target index. When using append-only indices it is better to use the create operation. This also applies to new append-only indexes, like data streams.
The scope of this improvement is to add a new boolean configuration option, append-only, to the Elasticsearch sink, with a default value of false (to keep the current behaivour) that when enabled makes it use the create operation instead of the index one when sending data.
