Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3152

Make index interval configuable when using MapFileOutputFormat for map-reduce job

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.16.0
    • 0.17.0
    • io
    • None
    • ubuntu, 64bit, jdk1.6

    • Reviewed
    • Add a static method MapFile#setIndexInterval(Configuration, int interval) so that MapReduce jobs that use MapFileOutputFormat can set the index interval.

    Description

      Per discussion with Doug Cutting on hadoop user mailing around Mar 21,
      thread title "MapFile and MapFileOutputFormat". Currently, there is no way
      to change the index interval for the output MapFile in a map-reduce job.
      As suggested, adding a static method MapFile(Configuration, int) to set the
      index interval and stores in Configuration, then MapFile.Writer constructor
      reads the setting from configuration may be a good idea.

      I also noticed that Hbase did similar things in HBASE-364.

      Attachments

        1. mapfile.diff
          2 kB
          Rong-En Fan
        2. mapfile.diff
          1 kB
          Rong-En Fan

        Activity

          People

            cutting Doug Cutting
            rafan Rong-En Fan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: