[STORM-3133] Extend metrics on Nimbus and LogViewer

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: storm-server
• Labels: pull-request-available
• Assignee: Zhengdai Hu
• Reporter: Zhengdai Hu
• Created: 29/Jun/18 17:55
• Updated: 10/Aug/18 17:13
• Resolved: 10/Aug/18 17:13

Description

Include but not limited to

Logviewer

1. Clean-up time
2. Time to complete one clean up loop Time.
3. Disk usage by logs before cleanup and After cleanup loop. ( Just like GC.?)
4. Failures/exceptions.
5. Search request Cnt: By category - Archived/non-archived
6. Search Request - Response time
7. Search Request - 0 result Cnt
8. Search Result - open files
9. File partial read count
10. File Download request Cnt/ And Size served
11. Disk IO by logviewer
12. CPU usage ( for unzipping files)

Nimbus Additional:

• Topology stormjar.ser/stormconf.ser/stormser.ser file upload time.
• Scheduler related metrics would be a long list generic and specific to different strategies.
• Most if not all cluster summary can be pushed as Metrics.
• Restart cnt
• Nimbus loss of leadership
• UI not responding (https://jira.ouroath.com/browse/YSTORM-4838)
• Negative resource scheduling events (https://jira.ouroath.com/browse/YSTORM-4940)
• Excessive scheduling time

Issue Links

• is a parent of: STORM-3144 Extend metrics on Nimbus
• relates to: STORM-3187 Nimbus code refactoring and cleanup
• relates to: STORM-3099 Extend metrics on supervisor, workers, and DRPC
• relates to: STORM-3186 Customizable configuration for metric reporting interval
• relates to: STORM-3188 Removing try-catch block from getAndResetWorkerHeartbeats
• links to: GitHub Pull Request #2754