[HADOOP-15125] Complete integration of new StorageStatistics

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.8.0, 2.9.0, 3.0.0
• Component/s: fs
• Target Version/s: 3.5.0
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 18/Dec/17 11:06
• Updated: 04/Jan/24 11:25

Description

HADOOP-13065 added the new StorageStatistics API. But there's a couple of subtasks remaining, and we are getting more experience of using it.

This JIRA covers the task of pulling those patches in, evolving what we have, targeting, realistically, v 3.2

Issue Links

• is related to: HDFS-13992 cross-cluster rack awareness for distcp

Sub-Tasks

• Refactor FileSystem$Statistics to use StorageStatistics: Open
• Rack-aware read bytes stats should be managed by HFDS specific StorageStatistics: Open
• Add thread local mechanism for aggregating file system storage stats: Patch Available
• Slow FileSystem.Statistics counters implementation: Patch Available
• Hadoop FS contract spec & tests to cover StorageStatistics: Open