Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-732

SequenceFile's header should allow to store metadata in the form of key/value pairs

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: io
    • Labels:
      None

      Description

      The sequence file currently stores a fixed list of metadata attributes, such as key/value class names,
      compression method, etc. To make sequence file more self descriptable, it should allow to store a list of key/value pairs. One particular attribute of interest is to indicate whether the key/value classes are actually hadoop record classes,
      if so, store the DDls for the records. This way, we may create tools to extract DDl from a sequence file and
      then generate necessary classes. It also make it possible to provide an interpretive version of Hadoop record.
      This way, even in the situation where Hadoop or the application does not have the necessary classes,
      a sequence file of Hadoop records can be read and deserialized "interpretively".

        Attachments

        1. seqFileMetadata.patch
          32 kB
          Runping Qi
        2. seqFileMetadata.patch.2
          24 kB
          Runping Qi

          Activity

            People

            • Assignee:
              runping Runping Qi
              Reporter:
              runping Runping Qi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: