[MAPREDUCE-5465] Tasks are often killed before they exit on their own - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.8.0, 3.0.0-alpha1
Component/s: mr-am, mrv2
Labels:
None

Hadoop Flags:

Reviewed

Description

If there is profiling enabled for mapper or reducer then hprof dumps profile.out at process exit. It is dumped after task signaled to AM that work is finished.

AM kills container with finished work without waiting for hprof to finish dumps. If hprof is dumping larger outputs (such as with depth=4 while depth=3 works) , it could not finish dump in time before being killed making entire dump unusable because cpu and heap stats are missing.

There needs to be better delay before container is killed if profiling is enabled.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

MAPREDUCE-5465.patch
10/Sep/13 22:06
3 kB
Ravi Prakash
MAPREDUCE-5465-2.patch
03/Mar/14 16:31
36 kB
Ming Ma
MAPREDUCE-5465-3.patch
04/Apr/14 19:29
41 kB
Ming Ma
MAPREDUCE-5465-4.patch
19/Apr/14 06:43
89 kB
Ming Ma
MAPREDUCE-5465-5.patch
22/Apr/14 18:45
90 kB
Ming Ma
MAPREDUCE-5465-6.patch
05/May/14 21:51
90 kB
Ming Ma
MAPREDUCE-5465-7.patch
29/Aug/14 02:46
86 kB
Ming Ma
MAPREDUCE-5465-8.patch
03/Sep/14 16:56
87 kB
Ming Ma
MAPREDUCE-5465-9.patch
02/Apr/15 21:47
87 kB
Ming Ma
MAPREDUCE-5465-branch-2.patch
11/May/15 20:23
86 kB
Ming Ma

Issue Links

is related to

MAPREDUCE-6735 Performance degradation caused by MAPREDUCE-5465 and HADOOP-12107

Open

Activity

People

Assignee:: Ming Ma

Reporter:: Radim Kolar

Votes:: 0 Vote for this issue

Watchers:: 15 Start watching this issue

Dates

Created:: 03/Mar/13 15:26

Updated:: 30/Aug/16 01:20

Resolved:: 11/May/15 22:41