-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: backlog
-
Fix Version/s: None
-
Component/s: tests
-
Labels:None
YARN-2410 highlights an interesting longevity bug, wherein it appears if there are large # of reducers ( > 6K )+ 40 map outputs per node, reducers all asking for mapper node outputs at a similar time can lead to scheduler crashing.
Would be great to see a workload that simulates this in the longevity tests.
- incorporates
-
MAPREDUCE-6474 ShuffleHandler can possibly exhaust nodemanager file descriptors
-
- Closed
-