[MAPREDUCE-353] Allow shuffle read and connection timeouts to be configurable - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.21.0
Fix Version/s: 0.21.0
Component/s: None
Labels:
None

Release Note:
Expert level config properties mapred.shuffle.connect.timeout and mapred.shuffle.read.timeout that are to be used at cluster level are added by this patch.

Description

It would be good for latency-sensitive applications to tune the shuffle read/connection timeouts... in fact this made a huge difference to terasort since we were seeing individual shuffles stuck for upwards of 60s and had to have a very small read timeout.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

patch-353-ydist.txt
10/Dec/09 04:08
3 kB
Jothi Padmanabhan
patch-353-ydist.txt
01/Dec/09 05:20
3 kB
Amareshwari Sriramadasu
MR-353.v1.patch
10/Jul/09 10:57
3 kB
Ravi Gummadi
MR-353.patch
08/Jul/09 04:28
3 kB
Ravi Gummadi

Issue Links

relates to

MAPREDUCE-1249 mapreduce.reduce.shuffle.read.timeout's default value should be 3 minutes, in mapred-default.xml

Closed

Activity

People

Assignee:: Ravi Gummadi

Reporter:: Arun Murthy

Votes:: 1 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 08/May/09 09:19

Updated:: 24/Aug/10 21:13

Resolved:: 16/Jul/09 09:45