[RANGER-3532] Delete Archived Spooled Audit Logs Based on TTL

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: audit
• Assignee: Unassigned
• Reporter: David Mollitor
• Created: 02/Dec/21 14:38
• Updated: 02/Dec/21 14:38

Description

As I understand it,...

When an audit destination (HDFS/SOLR) is offline, Ranger plugin can spool audit messages to the local disk.  Once the destination comes back online, the Ranger plugin will resume transmitting audit messages.  Once all audit messages are transmitted, the log file containing the message is sent to the audit 'archive' directory.  From there, if there are more than (configurable) 100 archived audit log files, then some number of files are deleted to bring that number down to 100.

 

This can be problematic if the number of audits is very large (and therefore spooled audit log files are very large) and they can sit in the archive directory for very long periods of time.  As I understand it, the only way for them to be deleted is if another outage event occurs and more files are created, always keeping the total number of files at 100.

 

Please add an additional criteria for deleting files: TTL

 

Delete archived audit files which are older than (configurable) a week.

Issue Links

• relates to: RANGER-2820 Difference between audit log spool directory and the archive directory under spool in Ranger