Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1463

[JAVA] Restructure ValueVector hierarchy to minimize compile-time generated code

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • Java

    Description

      The templates used in the java package are very high mainteance and the if conditions are hard to track. As started in the discussion here: https://github.com/apache/arrow/pull/1012, I'd like to propose that we modify the structure of the internal value vectors and code generation dynamics.

      Create new abstract base vectors:

      BaseFixedVector
      BaseVariableVector
      BaseNullableVector

      For each of these, implement all the basic functionality of a vector without using templating.

      Evaluate whether to use code generation to generate specific specializations of this functionality for each type where needed for performance purposes (probably constrained to mutator and accessor set/get methods). Giant and complex if conditions in the templates are actually worse from my perspective than a small amount of hand written duplicated code since templates are much harder to work with.

      Attachments

        Issue Links

          1.
          [JAVA] Document requirements and non/requirements for ValueVector updates Sub-task Resolved Siddharth Teotia Actions
          2.
          [JAVA] Design updated ValueVector Object Hierarchy Sub-task Resolved Siddharth Teotia Actions
          3.
          [JAVA] Create Prototype Code Hierarchy (Implementation Phase 1) Sub-task Resolved Siddharth Teotia Actions
          4.
          [JAVA] ValueVector hierarchy (Implementation Phase 2) Sub-task Resolved Siddharth Teotia Actions
          5.
          [JAVA] Create Benchmarking Suite for prototypes Sub-task Open Unassigned Actions
          6.
          [JAVA] Implement final ValueVector updates Sub-task Resolved Siddharth Teotia Actions
          7.
          [JAVA] Create Benchmarking Suite for final ValueVector updates Sub-task Open Unassigned Actions
          8.
          [Java] Remove non-nullable vectors in new vector class hierarchy Sub-task Resolved Bryan Cutler Actions
          9.
          [Java] Remove public static helper method in vector classes for JSONReader/Writer Sub-task Resolved Li Jin Actions
          10.
          [Java] Remove accessor/mutator Sub-task Resolved Li Jin Actions
          11.
          [Java] Integration test breaks without zeroing out validity vectors Sub-task Resolved Unassigned Actions
          12.
          [Java] Rename MapVector to StructVector Sub-task Resolved Bryan Cutler Actions
          13.
          [Java] Resolve new vector classes structure for timestamp, date and maybe interval Sub-task Closed Li Jin Actions
          14.
          [Java] Remove legacy vector classes Sub-task Resolved Bryan Cutler Actions
          15.
          [Java] Combine MapVector and NonNullableMapVector Classes Sub-task Closed Bryan Cutler Actions
          16.
          [Java] Add BitVector APIs from old vector class Sub-task Resolved Bryan Cutler Actions
          17.
          [Java] Change vector getMinorType to use MinorType instead of Types.MinorType Sub-task Resolved Bryan Cutler Actions
          18.
          [Java] Restore previous MapVector class names Sub-task Resolved Bryan Cutler Actions

          Activity

            People

              siddteotia Siddharth Teotia
              jnadeau Jacques Nadeau
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment