Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11283

Allow for Parent parameter for Parent/Child join - ElasticsearchIO

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P2
    • Resolution: Unresolved
    • Affects Version/s: 2.25.0
    • Fix Version/s: None
    • Component/s: io-java-elasticsearch
    • Labels:
      None

      Description

      It is impossible to index Parent/Child documents using ElasticsearchIO

       

      When using parent/child documents in Elasticsearch, child documents must be indexed using the "parent" attribute when using the bulk endpoint.

       

      https://www.elastic.co/guide/en/elasticsearch/guide/master/indexing-parent-child.html

      https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

       

      https://github.com/apache/beam/blob/master/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java

      There needs to be a "parent" attribute on the DocumentMetadata object. I think a FieldValueExtractFn would work really nicely for this

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                acuitymd Robert Coe
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 2h
                  2h
                  Remaining:
                  Remaining Estimate - 2h
                  2h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified