[MAPREDUCE-1100] User's task-logs filling up local disks on the TaskTrackers - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Won't Fix
Affects Version/s: 0.20.1, 0.20.2, 0.21.0
Fix Version/s: 0.21.1, 0.22.1
Component/s: tasktracker
Labels:
- critical-0.22.0

Target Version/s:

0.22.0

Description

Some user's jobs are filling up TT disks by outrageous logging. mapreduce.task.userlog.limit.kb is not enabled on the cluster. Disks are getting filled up before task-log cleanup via mapred.task.userlog.retain.hours can kick in.

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

reducetask-log-level.patch

15/Feb/11 01:37

4 kB

Todd Lipcon
patch-1100-fix-ydist.2.txt

10/Mar/10 10:10

13 kB

Vinod Kumar Vavilapalli
MAPREDUCE-1100-20091216.2.txt

16/Dec/09 18:18

53 kB

Arun C Murthy
MAPREDUCE-1100-20091106.txt

06/Nov/09 09:02

79 kB

Vinod Kumar Vavilapalli
MAPREDUCE-1100-20091102.txt

02/Nov/09 09:10

51 kB

Vinod Kumar Vavilapalli

Options
- Show All
- Show Open

Sub-Tasks

1.	Cleanup of task-logs should happen in TaskTracker instead of the Child		Closed	Amareshwari Sriramadasu
2.	Truncate logs of finished tasks to prevent node thrash due to excessive logging		Resolved	Vinod Kumar Vavilapalli

Activity

Ascending order - Click to sort in descending order

Vinod Kumar Vavilapalli created issue - 13/Oct/09 09:13

Vinod Kumar Vavilapalli made changes - 15/Oct/09 12:04

Field	Original Value	New Value
Assignee		Vinod K V [ vinodkv ]

Vinod Kumar Vavilapalli made changes - 02/Nov/09 09:10

Attachment

MAPREDUCE-1100-20091102.txt [ 12423815 ]

Vinod Kumar Vavilapalli made changes - 06/Nov/09 09:02

Attachment

MAPREDUCE-1100-20091106.txt [ 12424193 ]

Vinod Kumar Vavilapalli made changes - 19/Nov/09 13:05

Link

This issue incorporates ~~MAPREDUCE-927~~ [ ~~MAPREDUCE-927~~ ]

Arun C Murthy made changes - 16/Dec/09 18:18

Attachment

MAPREDUCE-1100-20091216.2.txt [ 12428200 ]

Vinod Kumar Vavilapalli made changes - 01/Mar/10 04:44

Link

This issue incorporates ~~MAPREDUCE-927~~ [ ~~MAPREDUCE-927~~ ]

Vinod Kumar Vavilapalli made changes - 10/Mar/10 10:10

Attachment

patch-1100-fix-ydist.2.txt [ 12438394 ]

Eli Collins made changes - 23/Sep/10 00:27

Fix Version/s		0.21.1 [ 12315272 ]
Fix Version/s		0.22.0 [ 12314184 ]
Affects Version/s		0.20.2 [ 12314205 ]
Affects Version/s		0.20.1 [ 12314047 ]

Eli Collins made changes - 16/Dec/10 18:13

Priority

Major [ 3 ]

Blocker [ 1 ]

Todd Lipcon made changes - 15/Feb/11 01:37

Attachment

reducetask-log-level.patch [ 12471047 ]

Konstantin Shvachko made changes - 24/Nov/11 01:14

Priority

Blocker [ 1 ]

Major [ 3 ]

Arun C Murthy made changes - 24/Nov/11 06:48

Target Version/s

0.22.0 [ 12314184 ]

Arun C Murthy made changes - 24/Nov/11 22:08

Labels

critical-0.22.0

Konstantin Shvachko made changes - 12/Dec/11 06:09

Fix Version/s		0.22.1 [ 12319242 ]
Fix Version/s	0.22.0 [ 12314184 ]

Allen Wittenauer made changes - 29/Jul/14 19:40

Status	Open [ 1 ]	Resolved [ 5 ]
Resolution		Won't Fix [ 2 ]

People

Assignee:

Vinod Kumar Vavilapalli

Reporter:

Vinod Kumar Vavilapalli

Votes:

0 Vote for this issue

Watchers:

25 Start watching this issue

Dates

Created:

13/Oct/09 09:13

Updated:

29/Jul/14 19:40

Resolved:

29/Jul/14 19:40

Development

Agile

View on Board