Issue Details (XML | Word | Printable)

Key: HADOOP-3152
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Minor Minor
Assignee: Doug Cutting
Reporter: Rong-En Fan
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Make index interval configuable when using MapFileOutputFormat for map-reduce job

Created: 02/Apr/08 04:39 AM   Updated: 21/May/08 08:06 PM
Return to search
Component/s: io
Affects Version/s: 0.16.0
Fix Version/s: 0.17.0

Time Tracking:
Not Specified

File Attachments:
  Size
File Licensed for inclusion in ASF works mapfile.diff 2008-04-04 07:03 AM Rong-En Fan 2 kB
File Licensed for inclusion in ASF works mapfile.diff 2008-04-02 04:40 AM Rong-En Fan 1 kB
Environment: ubuntu, 64bit, jdk1.6

Hadoop Flags: Reviewed
Release Note: Add a static method MapFile#setIndexInterval(Configuration, int interval) so that MapReduce jobs that use MapFileOutputFormat can set the index interval.
Resolution Date: 05/Apr/08 12:03 AM


 Description  « Hide
Per discussion with Doug Cutting on hadoop user mailing around Mar 21,
thread title "MapFile and MapFileOutputFormat". Currently, there is no way
to change the index interval for the output MapFile in a map-reduce job.
As suggested, adding a static method MapFile(Configuration, int) to set the
index interval and stores in Configuration, then MapFile.Writer constructor
reads the setting from configuration may be a good idea.

I also noticed that Hbase did similar things in HBASE-364.



 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Rong-En Fan made changes - 02/Apr/08 04:40 AM
Field Original Value New Value
Attachment mapfile.diff [ 12379100 ]
Doug Cutting made changes - 02/Apr/08 09:19 PM
Fix Version/s 0.17.0 [ 12312913 ]
Hadoop Flags [Reviewed]
Assignee Doug Cutting [ cutting ]
Status Open [ 1 ] Patch Available [ 10002 ]
Owen O'Malley made changes - 03/Apr/08 10:12 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Rong-En Fan made changes - 04/Apr/08 07:03 AM
Attachment mapfile.diff [ 12379358 ]
Doug Cutting made changes - 04/Apr/08 11:53 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Doug Cutting made changes - 05/Apr/08 12:03 AM
Resolution Fixed [ 1 ]
Release Note Add a static method MapFile#setIndexInterval(Configuration, int interval) so that MapReduce jobs that use MapFileOutputFormat can set the index interval.
Status Patch Available [ 10002 ] Resolved [ 5 ]
Doug Cutting made changes - 05/Apr/08 12:03 AM
Priority Major [ 3 ] Minor [ 4 ]
Issue Type Bug [ 1 ] New Feature [ 2 ]
Nigel Daley made changes - 21/May/08 08:06 PM
Status Resolved [ 5 ] Closed [ 6 ]