[BEAM-1637] Create Elasticsearch IO compatible with ES 5.x

• Type: New Feature
• Status: Resolved
• Priority: P3
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: io-java-elasticsearch
• Assignee: Etienne Chauchot
• Reporter: Etienne Chauchot
• Created: 07/Mar/17 14:35
• Updated: 16/May/20 13:18
• Resolved: 15/Sep/17 09:47

Description

The current Elasticsearch IO (see https://issues.apache.org/jira/browse/BEAM-425) is only compatible with Elasticsearch v 2.x. The aim is to have an IO compatible with ES v 5.x. Beyond being able to address v5.x elasticsearch instances, we could also leverage the use of the Elasticsearch pipeline API and also better split the dataset (be as close as possible of desiredBundleSize) thanks to the new ES split API that allows ES shards splitting.

Issue Links

• is cloned by: BEAM-3333 Create Elasticsearch IO compatible with ES 6.x
• is related to: BEAM-2410 Remove TransportClient from ElasticSearchIO to decouple IO and ES versions
• relates to: BEAM-1688 Getting listener timeout after waiting for [10000] ms from ElasticsearchIO
• links to: GitHub Pull Request #2230
• links to: GitHub Pull Request #3703

Sub-Tasks

• Elasticsearch IO - test ES 5?: Resolved
• Remove TransportClient from ElasticSearchIO to decouple IO and ES versions: Resolved