[NIFI-7346] Provenance Repository can stop producing data and become empty

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: Core Framework
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 09/Apr/20 12:46
• Updated: 15/Jun/20 18:09
• Resolved: 09/Apr/20 17:34

Description

The Provenance Repository is designed to periodically age off old data. This is done by looking at the Last Modified Time for all event files, and then removing any file whose Last Modified Time is longer than the configured "nifi.provenance.repository.max.storage.time" time.

However, it does this without first checking if the event file is the "active" event file. As a result, it can delete the event file while it is being written to. If this happens, on Linux and OSX, the delete will succeed and any further writes to the underlying file handle will simply disappear. And checking the size of the file will always return 0.

As a result, the Event File will never roll over because it never reaches the max size. And it doesn't roll over based on time due to NIFI-7339. As a result, all data will be aged off from the Provenance Repository and no new data will be added until restart.

Issue Links

• is duplicated by: NIFI-7337 Can't display any provenance data
• links to: GitHub Pull Request #4199