[HDFS-13055] Aggregate usage statistics from datanodes - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 3.2.0
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed

Description

We collect variety of statistics in DataNodes and expose them via JMX. Aggregating some of the high level statistics which we are already collecting in DataNodeMetrics (like bytesRead,bytesWritten etc) over a configurable time window will create a central repository accessible via JMX and UI.

Breaking this into 3 parts as suggested by arpitagarwal

Generate usage report on DN. (will handle in this jira)
Pass usage report to NN periodically.
Process report on NN and expose stats via JMX.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HDFS-13055.001.patch
24/Jan/18 02:11
58 kB
Ajay Kumar
HDFS-13055.002.patch
06/Feb/18 21:36
59 kB
Ajay Kumar
HDFS-13055.003.patch
22/Feb/18 21:57
59 kB
Ajay Kumar
HDFS-13055.004.patch
23/Feb/18 20:18
59 kB
Ajay Kumar
HDFS-13055.005.patch
27/Feb/18 18:01
61 kB
Ajay Kumar
HDFS-13055.006.patch
05/Mar/18 19:23
61 kB
Ajay Kumar
HDFS-13055.007.patch
15/Mar/18 02:13
18 kB
Ajay Kumar
HDFS-13055.008.patch
16/Mar/18 17:30
18 kB
Ajay Kumar
HDFS-13055.009.patch
20/Mar/18 00:30
18 kB
Ajay Kumar
HDFS-13055.010.patch
21/Mar/18 17:22
18 kB
Ajay Kumar

Issue Links

causes

HADOOP-15404 Remove multibyte characters in DataNodeUsageReportUtil

Resolved

links to

Review Board 65643

Activity

People

Assignee:: Ajay Kumar

Reporter:: Ajay Kumar

Votes:: 0 Vote for this issue

Watchers:: 9 Start watching this issue

Dates

Created:: 23/Jan/18 22:57

Updated:: 23/Apr/18 05:25

Resolved:: 22/Apr/18 15:13