Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-522

MapFile should support block compression

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: io
    • Labels:
      None

      Description

      MapFile is layered on SequenceFile and permits random-access to sorted data files (typically reduce output) through a parallel index file. This is used widely in Nutch (e.g. at search time for displaying cached pages, incoming links, etc). Such sorted data should benefit from block compression, but the current MapFile API does not support specification of block compression. Also, even if it did, the semantics of SequenceFile methods like seek() and getPosition() are changed under block compression so that MapFile may not work.

        Attachments

        1. block-compress-map-file.patch
          18 kB
          Doug Cutting

          Activity

            People

            • Assignee:
              cutting Doug Cutting
              Reporter:
              cutting Doug Cutting
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: