[METRON-1677] UUIDv4 GUID is not Lucene friendly

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Assignee: Unassigned
• Reporter: Ali Nazemian
• Created: 18/Jul/18 01:25
• Updated: 19/Dec/18 21:17
• Resolved: 19/Dec/18 21:17

Description

Using UUIDv4 by UUID.randomUUID() in Java is not Lucene friendly and impacts Elasticsearch and Solr indexing/search performance and makes it unpredictable sometimes.

http://blog.mikemccandless.com/2014/05/choosing-fast-unique-identifier-uuid.html

Moreover, specifying doc id at the client side will impact indexing throughput due to enabling Elasticsearch deduplication policy and changing insert to upsert. Hence, indexing throughput can be increased by providing an ability to disable ID generation at the client side. Currently, the way ID is generated can be overwritten at the config level by replacing Metron default guid via Stellar, but it is not possible to disable it completely to let Elasticsearch decide what ID can be used for the corresponding document.

Issue Links

• links to: GitHub Pull Request #1269

Sub-Tasks

• Allow Customization of Elasticsearch Document ID: Done
• Correct Test Data Load in Elasticsearch Integration Tests: Done
• Allow Elasticsearch to Auto-Generate the Document ID: Done