Hadoop Common
  1. Hadoop Common
  2. HADOOP-732

SequenceFile's header should allow to store metadata in the form of key/value pairs

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: io
    • Labels:
      None

      Description

      The sequence file currently stores a fixed list of metadata attributes, such as key/value class names,
      compression method, etc. To make sequence file more self descriptable, it should allow to store a list of key/value pairs. One particular attribute of interest is to indicate whether the key/value classes are actually hadoop record classes,
      if so, store the DDls for the records. This way, we may create tools to extract DDl from a sequence file and
      then generate necessary classes. It also make it possible to provide an interpretive version of Hadoop record.
      This way, even in the situation where Hadoop or the application does not have the necessary classes,
      a sequence file of Hadoop records can be read and deserialized "interpretively".

      1. seqFileMetadata.patch.2
        24 kB
        Runping Qi
      2. seqFileMetadata.patch
        32 kB
        Runping Qi

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Runping Qi
            Reporter:
            Runping Qi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development