[NIFI-2452] Provenance Repository's Index Readers can be prematurely closed

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.0.0, 1.0.0-Beta
• Component/s: Core Framework
• Assignee: Joe Witt
• Reporter: Mark Payne
• Created: 01/Aug/16 18:49
• Updated: 04/Aug/16 21:24
• Resolved: 03/Aug/16 17:07

Description

I occasionally see when I run Provenance queries against an active provenance repository that the JVM will crash, writing out an hs_err_pid_XXX.log file. This appears to be related to https://issues.apache.org/jira/browse/LUCENE-7183 which indicates that it's caused by using a closed IndexReader.

Adding to description from nice Github comments from mark:
Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient

In resolving the JIRA also cleaned up logic for handling heartbeat messages where a few states weren't accounted for leading to an incorrect accounting of heartbeats.

Issue Links

• links to: GitHub Pull Request #771