Hadoop Common
  1. Hadoop Common
  2. HADOOP-3152

Make index interval configuable when using MapFileOutputFormat for map-reduce job

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.17.0
    • Component/s: io
    • Labels:
      None
    • Environment:

      ubuntu, 64bit, jdk1.6

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a static method MapFile#setIndexInterval(Configuration, int interval) so that MapReduce jobs that use MapFileOutputFormat can set the index interval.

      Description

      Per discussion with Doug Cutting on hadoop user mailing around Mar 21,
      thread title "MapFile and MapFileOutputFormat". Currently, there is no way
      to change the index interval for the output MapFile in a map-reduce job.
      As suggested, adding a static method MapFile(Configuration, int) to set the
      index interval and stores in Configuration, then MapFile.Writer constructor
      reads the setting from configuration may be a good idea.

      I also noticed that Hbase did similar things in HBASE-364.

      1. mapfile.diff
        1 kB
        Rong-En Fan
      2. mapfile.diff
        2 kB
        Rong-En Fan

        Activity

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Rong-En Fan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development