[OAK-9015] ElasticSearchIndexWriter#close method implementation doesn't do what the underlying contract expects | BulkProcessor resource is never closed and it always returns false

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.30.0
• Assignee: Nitin Gupta
• Reporter: Nitin Gupta
• Created: 17/Apr/20 04:17
• Updated: 08/Jun/20 11:05
• Resolved: 21/Apr/20 12:09

Description

A new ElasticserachIndexWriter instance is created everytime the async indexer service runs and the close() method in the FullTextWriter interface has the following contract

 

/**
 * Closes the underlying resources.
 *
 * @param timestamp timestamp to be used for recording at status in NodeBuilder
 * @return true if index was updated or any write happened.
 */
boolean close(long timestamp) throws IOException;

However the implementation in ElasticserachIndexWriter returns false always and doesn't closes the BulkProcessor instances created.

 

The problem that needs to be handled here is that the bulk processor calls to elastic server are async in nature and the close method in ElasticserachIndexWriter could be called before the bulkProcessor completed it work. So we need to find a safe way to close the bulkProcessor and return true/false from ElasticserachIndexWriter#close() depending upon if the index was updated or not.

 

Looking at [0], both the bulkProcessor.close and awaitClose methods close the bulkPocessor after flushing all the documents that were loaded to the bulkProcessor till that moment. So we just need to make sure that, all the needed docs should be loaded in the bulkProcessor when ElasticserachIndexWriter#close() is called.

 

[0]: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk-processor.html