[YARN-276] Capacity Scheduler can hang when submit many jobs concurrently - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Duplicate
Affects Version/s: 2.0.1-alpha, 3.0.0-alpha1
Fix Version/s: None
Component/s: capacityscheduler
Labels:
- incompatible

Target Version/s:

2.1.0-beta

Description

In hadoop2.0.1,When i submit many jobs concurrently at the same time,Capacity scheduler can hang with most resources taken up by AM and don't have enough resources for tasks.And then all applications hang there.
The cause is that "yarn.scheduler.capacity.maximum-am-resource-percent" not check directly.Instead ,this property only used for maxActiveApplications. And maxActiveApplications is computed by minimumAllocation (not by Am actually used).

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

YARN-276.patch
18/Dec/12 02:38
8 kB
Nemon Lou
YARN-276.patch
18/Dec/12 07:11
10 kB
Nemon Lou
YARN-276.patch
19/Dec/12 00:40
11 kB
Nemon Lou
YARN-276.patch
19/Dec/12 09:23
13 kB
Nemon Lou
YARN-276.patch
20/Dec/12 04:07
13 kB
Nemon Lou
YARN-276.patch
24/Dec/12 12:30
25 kB
Nemon Lou
YARN-276.patch
13/Apr/13 05:22
14 kB
Nemon Lou
YARN-276.patch
15/Apr/13 07:58
30 kB
Nemon Lou
YARN-276.patch
15/Apr/13 08:08
30 kB
Nemon Lou
YARN-276.patch
15/Apr/13 08:45
31 kB
Nemon Lou
YARN-276.patch
15/Apr/13 08:47
31 kB
Nemon Lou
YARN-276.patch
15/Apr/13 11:08
31 kB
Nemon Lou
YARN-276.patch
23/Apr/13 08:58
45 kB
Nemon Lou
YARN-276.patch
04/Jun/13 05:09
45 kB
Nemon Lou
YARN-276.patch
05/Jun/13 11:35
44 kB
Nemon Lou
YARN-276.patch
14/Jun/13 06:54
54 kB
Nemon Lou

Issue Links

depends upon

YARN-1231 Fix test cases that will hit max- am-used-resources-percent limit after YARN-276

Resolved

Activity

People

Assignee:: Nemon Lou

Reporter:: Nemon Lou

Votes:: 0 Vote for this issue

Watchers:: 14 Start watching this issue

Dates

Created:: 18/Dec/12 02:36

Updated:: 12/May/16 18:30

Resolved:: 08/May/15 17:26

Time Tracking

Estimated:

24h

Remaining:

24h

Logged:

Not Specified