[MAPREDUCE-5027] Shuffle does not limit number of outstanding connections - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 2.0.3-alpha, 0.23.5
Fix Version/s: 0.23.7, 2.1.0-beta
Component/s: None
Labels:
None

Description

The ShuffleHandler does not have any configurable limits to the number of outstanding connections allowed. Therefore a node with many map outputs and many reducers in the cluster trying to fetch those outputs can exhaust a nodemanager out of file descriptors.

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

MAPREDUCE-5027-4.patch
05/Mar/13 20:52
10 kB
Robert Parker
MAPREDUCE-5027-b023-2.patch
05/Mar/13 20:38
11 kB
Robert Parker
MAPREDUCE-5027-b023.patch
04/Mar/13 16:18
10 kB
Robert Parker
MAPREDUCE-5027-3.patch
04/Mar/13 02:24
10 kB
Robert Parker
MAPREDUCE-5027-2.patch
01/Mar/13 02:10
11 kB
Robert Parker
MAPREDUCE-5027.patch
27/Feb/13 02:37
10 kB
Robert Parker
MAPREDUCE-5027.patch
27/Feb/13 00:02
10 kB
Robert Parker

Activity

People

Assignee:

Robert Parker

Reporter:

Jason Darrell Lowe

Votes:

0 Vote for this issue

Watchers:

10 Start watching this issue

Dates

Created:

25/Feb/13 20:47

Updated:

03/Sep/14 22:57

Resolved:

06/Mar/13 00:12