Issue Details (XML | Word | Printable)

Key: HADOOP-1965
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Amar Kamat
Reporter: Devaraj Das
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Handle map output buffers better

Created: 28/Sep/07 01:20 PM   Updated: 08/Jul/09 04:52 PM
Return to search
Component/s: None
Affects Version/s: 0.16.0
Fix Version/s: 0.16.0

Time Tracking:
Not Specified

File Attachments:
  File Name Date Attached Attached By Size
PDF File Licensed for inclusion in ASF works 1965_single_proc_150mb_gziped.pdf 2007-10-17 11:38 AM Amar Kamat 2 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-1.patch 2007-10-31 04:53 PM Amar Kamat 8 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-Benchmark.patch 2007-11-29 08:53 AM Amar Kamat 28 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-Benchmark.patch 2007-11-29 05:45 AM Amar Kamat 28 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-Benchmark.patch 2007-11-27 10:55 PM Amar Kamat 28 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-Benchmark.patch 2007-11-27 11:24 AM Amar Kamat 27 kB
Text File Licensed for inclusion in ASF works HADOOP-1965-Benchmark.patch 2007-11-22 12:21 PM Amar Kamat 24 kB
Text File Licensed for inclusion in ASF works HADOOP-2419.patch 2008-01-07 03:55 PM Amar Kamat 30 kB
Text File Licensed for inclusion in ASF works HADOOP-2419.patch 2007-12-25 06:42 AM Amar Kamat 34 kB
Text File Licensed for inclusion in ASF works HADOOP-2419.patch 2007-12-17 12:34 PM Amar Kamat 29 kB
Text File Licensed for inclusion in ASF works HADOOP-2419.patch 2007-12-17 05:05 AM Amar Kamat 29 kB
Image Attachments:

1. 1965_single_proc_150mb_gziped.jpeg
(81 kB)
2. 1965_single_proc_150mb_gziped_breakup.png
(5 kB)

Resolution Date: 11/Jan/08 10:05 PM


 Description  « Hide
Today, the map task stops calling the map method while sort/spill is using the (single instance of) map output buffer. One improvement that can be done to improve performance of the map task is to have another buffer for writing the map outputs to, while sort/spill is using the first buffer.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Amar Kamat made changes - 17/Oct/07 11:38 AM
Field Original Value New Value
Attachment 1965_single_proc_150mb_gziped.pdf [ 12367866 ]
Amar Kamat made changes - 23/Oct/07 09:13 AM
Attachment 1965_single_proc_150mb_gziped_breakup.png [ 12368202 ]
Amar Kamat made changes - 23/Oct/07 09:18 AM
Attachment 1965_single_proc_150mb_gziped.jpeg [ 12368204 ]
Amar Kamat made changes - 31/Oct/07 04:53 PM
Attachment HADOOP-1965-1.patch [ 12368774 ]
Amar Kamat made changes - 31/Oct/07 04:54 PM
Fix Version/s 0.16.0 [ 12312740 ]
Affects Version/s 0.16.0 [ 12312740 ]
Status Open [ 1 ] Patch Available [ 10002 ]
Arun C Murthy made changes - 07/Nov/07 09:14 AM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 22/Nov/07 12:21 PM
Attachment HADOOP-1965-Benchmark.patch [ 12370064 ]
Amar Kamat made changes - 27/Nov/07 11:24 AM
Attachment HADOOP-1965-Benchmark.patch [ 12370291 ]
Amar Kamat made changes - 27/Nov/07 10:55 PM
Attachment HADOOP-1965-Benchmark.patch [ 12370354 ]
Amar Kamat made changes - 27/Nov/07 10:56 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Amar Kamat made changes - 29/Nov/07 05:45 AM
Attachment HADOOP-1965-Benchmark.patch [ 12370500 ]
Amar Kamat made changes - 29/Nov/07 05:48 AM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 29/Nov/07 05:48 AM
Status Open [ 1 ] Patch Available [ 10002 ]
Amar Kamat made changes - 29/Nov/07 08:53 AM
Attachment HADOOP-1965-Benchmark.patch [ 12370513 ]
Amar Kamat made changes - 29/Nov/07 08:57 AM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 29/Nov/07 08:58 AM
Status Open [ 1 ] Patch Available [ 10002 ]
Devaraj Das made changes - 29/Nov/07 10:04 AM
Resolution Fixed [ 1 ]
Status Patch Available [ 10002 ] Resolved [ 5 ]
Devaraj Das made changes - 14/Dec/07 07:40 PM
Status Resolved [ 5 ] Reopened [ 4 ]
Resolution Fixed [ 1 ]
Amar Kamat made changes - 17/Dec/07 05:05 AM
Attachment HADOOP-2419.patch [ 12371775 ]
Amar Kamat made changes - 17/Dec/07 05:07 AM
Status Reopened [ 4 ] Patch Available [ 10002 ]
Amar Kamat made changes - 17/Dec/07 12:33 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 17/Dec/07 12:34 PM
Attachment HADOOP-2419.patch [ 12371797 ]
Amar Kamat made changes - 17/Dec/07 12:36 PM
Fix Version/s 0.16.0 [ 12312740 ]
Fix Version/s 0.15.2 [ 12312877 ]
Affects Version/s 0.16.0 [ 12312740 ]
Status Open [ 1 ] Patch Available [ 10002 ]
Affects Version/s 0.15.2 [ 12312877 ]
Amar Kamat made changes - 18/Dec/07 04:49 AM
Fix Version/s 0.16.0 [ 12312740 ]
Fix Version/s 0.15.2 [ 12312877 ]
Affects Version/s 0.15.2 [ 12312877 ]
Affects Version/s 0.16.0 [ 12312740 ]
Arun C Murthy made changes - 19/Dec/07 08:09 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 25/Dec/07 06:42 AM
Attachment HADOOP-2419.patch [ 12372181 ]
Amar Kamat made changes - 25/Dec/07 06:45 AM
Status Open [ 1 ] Patch Available [ 10002 ]
Devaraj Das made changes - 04/Jan/08 01:33 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Amar Kamat made changes - 07/Jan/08 03:55 PM
Attachment HADOOP-2419.patch [ 12372632 ]
Amar Kamat made changes - 07/Jan/08 03:57 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Arun C Murthy made changes - 11/Jan/08 10:05 PM
Resolution Fixed [ 1 ]
Status Patch Available [ 10002 ] Resolved [ 5 ]
Nigel Daley made changes - 08/Feb/08 11:37 PM
Status Resolved [ 5 ] Closed [ 6 ]
Owen O'Malley made changes - 08/Jul/09 04:52 PM
Component/s mapred [ 12310690 ]

Powered by a free Atlassian JIRA open source license for Apache Software Foundation. Try JIRA - bug tracking software for your team.
Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators