[HAMA-1001] System and Processing Monitoring

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Component/s: network monitoring
• Assignee: Unassigned
• Reporter: ByungSeok Min
• Created: 07/Dec/17 01:20
• Updated: 12/Mar/19 19:33

Description

When using Hama on a large system,
It is very useful if we can know the processing status.

I would like to add the following features.
1. Real-time Usage of system resources
    - CPU, Memory, IO, etc.
2. Real-time processing performance
3. Real time processing status

 

Update:

It's nice to have step internal info observable. Basically we should emit as many useful stats as possible as we can.

 

4. Step stats info [1]

 

[1]. https://issues.apache.org/jira/browse/HAMA-1009