[HDFS-9395] Make HDFS audit logging consistant

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha1
• Hadoop Flags: Incompatible change, Reviewed

Release Note

<!-- markdown -->

Audit logs will now only be generated in the following two cases:
* When an operation results in an `AccessControlException`
* When an operation is successful

Notably, this means audit log events will not be generated for exceptions besides AccessControlException.

• Assignee: Kuhu Shukla
• Reporter: Kihwal Lee
• Created: 06/Nov/15 20:21
• Updated: 30/Aug/16 13:56
• Resolved: 26/Feb/16 15:47

Description

So, the big question here is what should go in the audit log? All failures, or just "permission denied" failures? Or, to put it a different way, if someone attempts to do something and it fails because a file doesn't exist, is that worth an audit log entry?

We are currently inconsistent on this point. For example, concat, getContentSummary, addCacheDirective, and setErasureEncodingPolicy create an audit log entry for all failures, but setOwner, delete, and setAclEntries attempt to only create an entry for AccessControlException-based failures. There are a few operations, like allowSnapshot, disallowSnapshot, and startRollingUpgrade that never create audit log failure entries at all. They simply log nothing for any failure, and log success for a successful operation.

So to summarize, different HDFS operations currently fall into 3 categories:
1. audit-log all failures
2. audit-log only AccessControlException failures
3. never audit-log failures

Which category is right? And how can we fix the inconsistency

Issue Links

• duplicates: HDFS-5730 Inconsistent Audit logging for HDFS APIs
• is related to: HDFS-9855 Modify TestAuditLoggerWithCommands to workaround the absence of HDFS-8332