Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-732

SequenceFile's header should allow to store metadata in the form of key/value pairs

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0
    • io
    • None

    Description

      The sequence file currently stores a fixed list of metadata attributes, such as key/value class names,
      compression method, etc. To make sequence file more self descriptable, it should allow to store a list of key/value pairs. One particular attribute of interest is to indicate whether the key/value classes are actually hadoop record classes,
      if so, store the DDls for the records. This way, we may create tools to extract DDl from a sequence file and
      then generate necessary classes. It also make it possible to provide an interpretive version of Hadoop record.
      This way, even in the situation where Hadoop or the application does not have the necessary classes,
      a sequence file of Hadoop records can be read and deserialized "interpretively".

      Attachments

        1. seqFileMetadata.patch
          32 kB
          Runping Qi
        2. seqFileMetadata.patch.2
          24 kB
          Runping Qi

        Activity

          People

            runping Runping Qi
            runping Runping Qi
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: