[CAMEL-8149] Support application-generated document identifiers in bulk index requests

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.14.0
• Fix Version/s: 2.15.0
• Component/s: camel-elasticsearch
• Patch Info: Patch Available
• Estimated Complexity: Novice
• Assignee: Claus Ibsen
• Reporter: Derek Abdine
• Created: 12/Dec/14 02:04
• Updated: 23/Dec/14 16:52
• Resolved: 23/Dec/14 10:07

Description

Elasticsearch (via the elasticsearch-java transport client) provides two categories of APIs to write and read data: Individual requests (index, get, delete) and bulk requests.

When performing bulk updates one creates individual index requests and adds them to the bulk request. When creating an index request one can set the source document, id, etc.

The current design of the camel-elasticsearch component controls the transformation and assembly of an input body (json string, byte[], xcontentfactory, map) to an index request. Thus, it is impossible to set the id on the index request that goes into a bulk action. The end result is that the id is set by the default behavior of the underlying elasticsearch-java client which generates a random identifier. This is problematic in situations where control is needed over the id, e.g. for de-duplication purposes.

My proposal is to improve the design of the producer to allow for elasticsearch-java ActionRequest sub-classes in the message body so that upstream message processors can control the creation of those requests.

I've attached a patch and sent a pull request on github.

Thank you!
Derek Abdine