Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3585

Integrate Trevni as another columnar oriented file format

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 0.10.0
    • None
    • Serializers/Deserializers
    • None

    Description

      add new avro module trevni as another columnar format.New columnar format need a columnar SerDe,seems fastutil is a good choice.the shark project use fastutil library as columnar serde library but it seems too large (almost 15m) for just a few primitive array collection.

      Attachments

        1. futurama_episodes.avro
          3 kB
          Mark Wagner
        2. HIVE-3585.1.patch.txt
          55 kB
          Mark Wagner
        3. HIVE-3585.2.patch.txt
          77 kB
          Sean Busbey
        4. HIVE-3585.3.patch.txt
          77 kB
          Edward Capriolo

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-5302 PartitionPruner logs warning on Avro non-partitioned data

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-3953 Reading of partitioned Avro data fails because of missing properties

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-4789 FetchOperator fails on partitioned Avro data

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. HIVE-895 Add SerDe for Avro serialized data

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. AVRO-1229 Trevni should support Boolean fields

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. FLUME-1669 Add support for columnar event serializer in HDFS

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. AVRO-806 add a column-major file format

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. AVRO-1241 improve trevni performance on string deserialization

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. PIG-3015 Rewrite of AvroStorage

          • Major - Major loss of function.
          • Closed
          There are no Sub-Tasks for this issue.

          Activity

            People

              mwagner Mark Wagner
              gemini5201314 alex gemini
              Votes:
              14 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

                Created:
                Updated: