Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5988

ElasticsearchIO listener timeout

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: io-java-elasticsearch
    • Labels:
      None

      Description

      Still getting a listener timeout when trying to insert a large amount of documents.

      ```

      java.io.IOException: listener timeout after waiting for [30000] ms
      at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:660)
      at org.elasticsearch.client.RestClient.performRequest(RestClient.java:219)
      at org.elasticsearch.client.RestClient.performRequest(RestClient.java:191)
      at org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn.flushBatch(ElasticsearchIO.java:1226)
      at org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn.finishBundle(ElasticsearchIO.java:1200)
      ```

      The listener timeouts are not handled in the underlying ES RestClient.

      A possible solution would be to catch these exceptions and include them in the retry handling mechanism

        Attachments

          Activity

            People

            • Assignee:
              wouts Wout Scheepers
              Reporter:
              wouts Wout Scheepers
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: