Issue Details (XML | Word | Printable)

Key: HADOOP-3366
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Arun C Murthy
Reporter: Arun C Murthy
Votes: 0
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Shuffle/Merge improvements

Created: 08/May/08 07:14 AM   Updated: 08/Jul/09 04:52 PM
Return to search
Component/s: None
Affects Version/s: None
Fix Version/s: 0.18.0

Time Tracking:
Not Specified

File Attachments:
  File Name Date Attached Attached By Size
Text File Licensed for inclusion in ASF works 3366.1.patch 2008-05-16 01:15 PM Devaraj Das 9 kB
Text File Licensed for inclusion in ASF works 3366.1.patch 2008-05-16 01:09 PM Devaraj Das 9 kB
Text File Licensed for inclusion in ASF works 3366.reducetask.patch 2008-06-06 02:27 PM Devaraj Das 3 kB
Text File Licensed for inclusion in ASF works HADOOP-3366_0_20080605.patch 2008-06-06 06:59 AM Arun C Murthy 10 kB
Text File Licensed for inclusion in ASF works HADOOP-3366_2_20080606.patch 2008-06-06 11:48 PM Arun C Murthy 16 kB
Text File Licensed for inclusion in ASF works HADOOP-3366_2_20080606.patch 2008-06-06 11:20 PM Arun C Murthy 15 kB
Text File ifile.patch 2008-05-19 11:06 AM Arun C Murthy 9 kB
Issue Links:
dependent
 

Release Note: Improved shuffle so that all fetched map-outputs are kept in-memory before being merged by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle.
Resolution Date: 07/Jun/08 12:09 AM


 Description  « Hide
This is intended to be a meta-issue to track various improvements to shuffle/merge in the reducer.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Arun C Murthy made changes - 08/May/08 07:15 AM
Field Original Value New Value
Link This issue depends upon HADOOP-2095 [ HADOOP-2095 ]
Arun C Murthy made changes - 08/May/08 07:15 AM
Link This issue depends upon HADOOP-3297 [ HADOOP-3297 ]
Arun C Murthy made changes - 08/May/08 07:15 AM
Link This issue depends upon HADOOP-3365 [ HADOOP-3365 ]
Devaraj Das made changes - 16/May/08 01:09 PM
Attachment 3366.1.patch [ 12382167 ]
Devaraj Das made changes - 16/May/08 01:15 PM
Attachment 3366.1.patch [ 12382170 ]
Arun C Murthy made changes - 19/May/08 11:06 AM
Attachment ifile.patch [ 12382292 ]
Arun C Murthy made changes - 06/Jun/08 06:59 AM
Attachment HADOOP-3366_0_20080605.patch [ 12383540 ]
Devaraj Das made changes - 06/Jun/08 02:27 PM
Attachment 3366.reducetask.patch [ 12383563 ]
Arun C Murthy made changes - 06/Jun/08 11:20 PM
Attachment HADOOP-3366_2_20080606.patch [ 12383609 ]
Arun C Murthy made changes - 06/Jun/08 11:48 PM
Attachment HADOOP-3366_2_20080606.patch [ 12383611 ]
Arun C Murthy made changes - 06/Jun/08 11:53 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Arun C Murthy made changes - 07/Jun/08 12:09 AM
Status Patch Available [ 10002 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Arun C Murthy made changes - 20/Jun/08 10:40 PM
Release Note Shuffle has been improved to ensure that all fetched map-outputs are kept in-memory before being merged. This is accomplished by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle.
Robert Chansler made changes - 22/Jul/08 05:52 PM
Release Note Shuffle has been improved to ensure that all fetched map-outputs are kept in-memory before being merged. This is accomplished by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle. Improved shuffle so that all fetched map-outputs are kept in-memory before being merged by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle.
Nigel Daley made changes - 22/Aug/08 07:50 PM
Status Resolved [ 5 ] Closed [ 6 ]
Owen O'Malley made changes - 08/Jul/09 04:52 PM
Component/s mapred [ 12310690 ]

Powered by a free Atlassian JIRA open source license for Apache Software Foundation. Try JIRA - bug tracking software for your team.
Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators