[BEAM-3026] Improve retrying in ElasticSearch client

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: io-java-elasticsearch
• Assignee: Ravi Pathak
• Reporter: Tim Robertson
• Created: 06/Oct/17 09:36
• Updated: 16/May/20 13:50
• Resolved: 24/Aug/18 13:22

Description

Currently an overloaded ES server will result in clients failing fast.
I suggest implementing backoff pauses. Perhaps something like this:

    ElasticsearchIO.ConnectionConfiguration conn = ElasticsearchIO.ConnectionConfiguration
      .create(new String[]{"http://...:9200"}, "test", "test")
      .retryWithWaitStrategy(WaitStrategies.exponentialBackoff(1000, TimeUnit.MILLISECONDS)
      .retryWithStopStrategy(StopStrategies.stopAfterAttempt(10)
    );

Issue Links

• links to: GitHub Pull Request #6146