[BEAM-11283] Allow for Parent parameter for Parent/Child join - ElasticsearchIO

• Type: Improvement
• Status: Open
• Priority: P2
• Resolution: Unresolved
• Affects Version/s: 2.25.0
• Component/s: io-java-elasticsearch
• Assignee: Unassigned
• Reporter: Robert Coe
• Created: 17/Nov/20 17:41
• Updated: 16/Mar/21 04:48

Description

It is impossible to index Parent/Child documents using ElasticsearchIO

 

When using parent/child documents in Elasticsearch, child documents must be indexed using the "parent" attribute when using the bulk endpoint.

 

https://www.elastic.co/guide/en/elasticsearch/guide/master/indexing-parent-child.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

 

https://github.com/apache/beam/blob/master/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java

There needs to be a "parent" attribute on the DocumentMetadata object. I think a FieldValueExtractFn would work really nicely for this

 

Issue Links

• is duplicated by: BEAM-11284 Allow for Parent parameter for Parent/Child join - ElasticsearchIO