[#HADOOP-872] map output sorter doesn't compress the outputs before the sort

Hadoop Common

map output sorter doesn't compress the outputs before the sort

Created: 09/Jan/07 11:07 PM Updated: 08/Jul/09 04:52 PM

Return to search

Component/s:

None

Affects Version/s:

0.10.0

Fix Version/s:

None

Time Tracking:

Not Specified

Issue Links:

Incorporates

This issue is part of:

~~HADOOP-2919~~ Create fewer copies of buffer data during sort/spill

Resolution Date:

31/Mar/08 10:51 PM

Description

« Hide

The map outputs are not compressed for the in memory sort, even if mapred.compress.map.output is set. This can significantly increase the in memory size of the buffered outputs.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Chris Douglas made changes - 04/Mar/08 09:45 PM

Field	Original Value	New Value
Link		This issue is part of ~~HADOOP-2919~~ [ ~~HADOOP-2919~~ ]

[ Permlink | « Hide ]

Owen O'Malley added a comment - 31/Mar/08 10:51 PM

~~HADOOP-2919~~ fixed this.

Owen O'Malley made changes - 31/Mar/08 10:51 PM

Resolution		Duplicate [ 3 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Owen O'Malley made changes - 31/Mar/08 10:52 PM

Fix Version/s

0.17.0 [ 12312913 ]

Nigel Daley made changes - 17/Apr/08 05:21 AM

Fix Version/s

0.17.0 [ 12312913 ]

Nigel Daley made changes - 17/Apr/08 05:26 AM

Status

Resolved [ 5 ]

Closed [ 6 ]

Owen O'Malley made changes - 08/Jul/09 04:52 PM

Component/s

mapred [ 12310690 ]