[YARN-2314] ContainerManagementProtocolProxy can create thousands of threads for a large cluster - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: 2.1.0-beta
Fix Version/s: 2.6.0
Component/s: client
Labels:
None

Target Version/s:

2.6.0
Hadoop Flags:

Reviewed

Description

ContainerManagementProtocolProxy has a cache of NM proxies, and the size of this cache is configurable. However the cache can grow far beyond the configured size when running on a large cluster and blow AM address/container limits. More details in the first comment.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

YARN-2314v2.patch
15/Oct/14 21:12
12 kB
Jason Darrell Lowe
YARN-2314.patch
14/Oct/14 19:39
12 kB
Jason Darrell Lowe
tez-yarn-2314.xlsx
16/Oct/14 09:31
64 kB
Rajesh Balamohan
nmproxycachefix.prototype.patch
22/Jul/14 22:00
3 kB
Jason Darrell Lowe
disable-cm-proxy-cache.patch
12/Sep/14 18:40
7 kB
Jason Darrell Lowe

Issue Links

is related to

MAPREDUCE-3333 MR AM for sort-job going out of memory

Closed

Activity

People

Assignee:: Jason Darrell Lowe

Reporter:: Jason Darrell Lowe

Votes:: 1 Vote for this issue

Watchers:: 25 Start watching this issue

Dates

Created:: 17/Jul/14 18:12

Updated:: 01/Dec/14 03:09

Resolved:: 25/Oct/14 06:07