[GOBBLIN-1869] ORCWriter should emit events to capture metrics on write side

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-core
• Assignee: Abhishek Tiwari
• Reporter: William Lo
• Created: 04/Aug/23 20:33
• Updated: 11/Aug/23 17:04

Description

Currently the metrics for streaming mainly relate to the Kafka consumption side. We want to also capture metrics through events from the ORCWriter to capture the memory usage and internal performance of the writer.