[YARN-8925] Updating distributed node attributes only when necessary - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.2.1
Fix Version/s: 3.3.0, 3.2.1
Component/s: resourcemanager
Labels:
- performance

Hadoop Flags:

Reviewed

Description

Currently if distributed node attributes exist, even though there is no change, updating for distributed node attributes will happen in every heartbeat between NM and RM. Updating process will hold NodeAttributesManagerImpl#writeLock and may have some influence in a large cluster. We have found nodes UI of a large cluster is opened slowly and most time it's waiting for the lock in NodeAttributesManagerImpl. I think this updating should be called only when necessary to enhance the performance of related process.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

YARN-8925.001.patch
22/Oct/18 07:12
12 kB
Tao Yang
YARN-8925.002.patch
02/Nov/18 08:18
69 kB
Tao Yang
YARN-8925.003.patch
05/Nov/18 08:48
82 kB
Tao Yang
YARN-8925.004.patch
08/Nov/18 08:17
86 kB
Tao Yang
YARN-8925.005.patch
10/Nov/18 08:22
86 kB
Tao Yang
YARN-8925.006.patch
15/Nov/18 03:10
86 kB
Tao Yang
YARN-8925.007.patch
16/Nov/18 08:22
86 kB
Tao Yang
YARN-8925.008.patch
20/Nov/18 07:06
87 kB
Tao Yang
YARN-8925.009.patch
20/Nov/18 08:02
87 kB
Tao Yang
YARN-8925.010.patch
11/Dec/18 09:25
91 kB
Tao Yang
YARN-8925-branch-3.2.001.patch
21/Dec/18 05:20
91 kB
Tao Yang

Issue Links

duplicates

YARN-7877 Node attributes error handling based on RM validation response

Resolved

Activity

People

Assignee:: Tao Yang

Reporter:: Tao Yang

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 22/Oct/18 06:55

Updated:: 20/Mar/19 16:59

Resolved:: 21/Dec/18 08:37