[NIFI-527] Persistent Prov Repo should compress write-ahead-log files in chunks

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.1.0
• Component/s: Extensions
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 20/Apr/15 17:36
• Updated: 28/Apr/15 13:31
• Resolved: 28/Apr/15 13:31

Description

Currently when we rollover a prov log, we compress the entire thing. This means that when we want to jump to a particular offset we have to open a GZIPInputStream and read through all of the data. If we instead compress the logs in chunks, we can actually jump to a particular chunk using FileInputStream.skip and then open a GZIPInputStream from there. Currently, this is by far the biggest bottleneck in the prov repo when doing queries.