[HUDI-1707] Improve Logging subsystem

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.10.0
• Component/s: code-quality
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Volodymyr Burenin
• Created: 19/Mar/21 19:26
• Updated: 08/Dec/21 07:00

Description

Currently Hudi has a relatively verbose logging on info level that is not particularly useful. Like latency measurements of file system views, print out of commit timelines, etc that could be super large, etc.

Additionally to that, the logging subsystem is suboptimal as it formats all messages before they are being passed into the logger, so it doesn't matter if logger will or will not print out a log message a lot of work is still being done anyway.

Would be also nice to add more messages on info level to determine at which phase Hudi is: ideally, the info level should be limited to the point that just looking into the logs should be clear enough at which phase Hudi is and what it is doing without being too verbose.

TBD: Add more thoughts on logging subsystem

Issue Links

• links to: GitHub Pull Request #2714