[NIFI-8691] Retention period per process group / processor

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.13.2
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Jul Tomten
• Created: 12/Jun/21 11:39
• Updated: 12/Jun/21 11:39

Description

Today retention periods are set globally in the properites file.

 

It's necessary to set retention periods per process group or individual processor.

 

For some scenarios it's necessary to keep files and data for many years.

For some scenarios it's necessary to delete files and data immediately.
Everything in between the above.

The reasons may be many like European GDPR law, tax regulations and laws, troubleshooting and many others.

 

NiFi isn't an archiving and storage system of course 
but for many scenarios it would make sense to have a fine control of the retention periods in NiFi.