Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-941

Enhancements to Hadoop record I/O - Part 1

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.12.0
    • Component/s: record
    • Labels:
      None
    • Environment:

      All

      Description

      Hadoop record I/O can be used effectively outside of Hadoop. It would increase its utility if developers can use it without having to import hadoop classes, or having to depend on Hadoop jars. Following changes to the current translator and runtime are proposed.

      Proposed Changes:

      1. Use java.lang.String as a native type for ustring (instead of Text.)
      2. Provide a Buffer class as a native Java type for buffer (instead of BytesWritable), so that later BytesWritable could be implemented as following DDL:
      module org.apache.hadoop.io {
      record BytesWritable

      { buffer value; }

      }
      3. Member names in generated classes should not have prefixes 'm' before their names. In the above example, the private member name would be 'value' not 'mvalue' as it is done now.
      4. Convert getters and setters to have CamelCase. e.g. in the above example the getter will be:
      public Buffer getValue();
      5. Generate clone() methods for records in Java i.e. the generated classes should implement Cloneable.
      6. Make generated Java codes for maps and vectors use Java generics.

      These are the proposed user-visible changes. Internally, the translator will be restructured so that it is easier to plug-in translators for different targets.

        Attachments

        1. jute-patch.txt
          249 kB
          Milind Bhandarkar

          Activity

            People

            • Assignee:
              milindb Milind Bhandarkar
              Reporter:
              milindb Milind Bhandarkar
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: