[HDDS-3694] Reduce dn-audit log

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Labels: Triaged performance pull-request-available
• Target Version/s: 1.0.0
• Assignee: Dinesh Chitlangia
• Reporter: Rajesh Balamohan
• Created: 01/Jun/20 10:37
• Updated: 03/Jun/20 15:49
• Resolved: 03/Jun/20 15:01

Description

Do we really need such fine grained audit log? This ends up creating too many entries for chunks.

2020-05-31 23:31:48,477 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324230275483 bcsId: 93943} | ret=SUCCESS |
2020-05-31 23:31:48,482 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267323565871437 bcsId: 93940} | ret=SUCCESS |
2020-05-31 23:31:48,487 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324230275483 bcsId: 93943} | ret=SUCCESS |
2020-05-31 23:31:48,497 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267324172472725 bcsId: 93934} | ret=SUCCESS |
2020-05-31 23:31:48,501 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267323675906396 bcsId: 93958} | ret=SUCCESS |
2020-05-31 23:31:48,504 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324230275483 bcsId: 93943} | ret=SUCCESS |
2020-05-31 23:31:48,509 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267323685343583 bcsId: 93974} | ret=SUCCESS |
2020-05-31 23:31:48,512 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267324172472725 bcsId: 93934} | ret=SUCCESS |
2020-05-31 23:31:48,516 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324332380586 bcsId: 0} | ret=SUCCESS |
2020-05-31 23:31:48,726 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267324232634780 bcsId: 93964} | ret=SUCCESS |
2020-05-31 23:31:48,733 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267323976323460 bcsId: 93967} | ret=SUCCESS |
2020-05-31 23:31:48,740 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324131512723 bcsId: 93952} | ret=SUCCESS |
2020-05-31 23:31:48,752 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267324230275483 bcsId: 93943} | ret=SUCCESS |
2020-05-31 23:31:48,760 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 165 locID: 104267323675906396 bcsId: 93958} | ret=SUCCESS |
2020-05-31 23:31:48,772 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 166 locID: 104267323685343583 bcsId: 93974} | ret=SUCCESS |
2020-05-31 23:31:48,780 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 164 locID: 104267324304724389 bcsId: 0} | ret=SUCCESS |
2020-05-31 23:31:48,787 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 164 locID: 104267323991724421 bcsId: 93970} | ret=SUCCESS |
2020-05-31 23:31:48,794 | INFO  | DNAudit | user=null | ip=null | op=WRITE_CHUNK {blockData=conID: 164 locID: 104267323725189479 bcsId: 93963} | ret=SUCCESS |

 

And ends up choking disk utilization with lesser write/mb/sec.

Refer to 100+ writes being written with 0.52 MB/sec and choking entire disk utilization.

 

Also, the username and IP are currently set as null. This needs to be replaced by using details from grpc

Issue Links

• relates to: HDDS-3706 Fetch user name & IP from gRPC for audit log in DN
• links to: GitHub Pull Request #1001