Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11276

Write with custom indexName and docId not present in document itself

    XMLWordPrintableJSON

    Details

    • Type: Wish
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: 2.25.0
    • Fix Version/s: None
    • Component/s: io-java-elasticsearch
    • Labels:
      None
    • Environment:
      Java, Google Dataflow

      Description

      ElasticIO connector write method is taking currently as input a "PCollection<String>" element ; string value will be passed as ElasticSearch document body. With methods withIdFn, and withIndexFn, we can overload Elastic document id and index name at runtime.

      There is some kind of situation where indexName and/or docId are not present in document itself (for example from Google Pubsub metadata or other).

      Is there a way to implement this usecase with current ElasticIO ? Otherwise, if it makes sense for you, what kind of improvment can be performed on connector ?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lionelsetan Lionel Setan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: