Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-352

Make Hive support column based storage

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.4.0
    • None
    • None
    • Reviewed
    • HIVE-352. Column-based storage format RCFile. (Yongqiang He via zshao)

    Description

      column based storage has been proven a better storage layout for OLAP.
      Hive does a great job on raw row oriented storage. In this issue, we will enhance hive to support column based storage.
      Acctually we have done some work on column based storage on top of hdfs, i think it will need some review and refactoring to port it to Hive.

      Any thoughts?

      Attachments

        1. 4-22 performace2.txt
          8 kB
          He Yongqiang
        2. 4-22 performance.txt
          6 kB
          He Yongqiang
        3. 4-22 progress.txt
          3 kB
          He Yongqiang
        4. hive-352-2009-4-15.patch
          126 kB
          He Yongqiang
        5. hive-352-2009-4-16.patch
          106 kB
          He Yongqiang
        6. hive-352-2009-4-17.patch
          109 kB
          He Yongqiang
        7. hive-352-2009-4-19.patch
          129 kB
          He Yongqiang
        8. hive-352-2009-4-22.patch
          138 kB
          He Yongqiang
        9. hive-352-2009-4-22-2.patch
          138 kB
          He Yongqiang
        10. hive-352-2009-4-23.patch
          139 kB
          He Yongqiang
        11. hive-352-2009-4-27.patch
          142 kB
          He Yongqiang
        12. hive-352-2009-4-30-2.patch
          146 kB
          He Yongqiang
        13. hive-352-2009-4-30-3.patch
          146 kB
          He Yongqiang
        14. hive-352-2009-4-30-4.patch
          156 kB
          He Yongqiang
        15. hive-352-2009-5-1.patch
          155 kB
          He Yongqiang
        16. hive-352-2009-5-1-3.patch
          158 kB
          He Yongqiang
        17. HIve-352-draft-2009-03-28.patch
          51 kB
          He Yongqiang
        18. Hive-352-draft-2009-03-30.patch
          60 kB
          He Yongqiang

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-461 Optimize RCFile reading by using column pruning results

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-337 LazySimpleSerDe should support multi-level nested array, map, struct types

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-360 Generalize the FileFormat Interface in Hive

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. AVRO-806 add a column-major file format

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-2097 Explore mechanisms for better compression with RC Files

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-460 Improve ColumnPruner to prune more aggressively and keep column information for input tables

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-756 performance improvement for RCFile and ColumnarSerDe in Hive

          • Major - Major loss of function.
          • Closed

          Activity

            People

              he yongqiang He Yongqiang
              he yongqiang He Yongqiang
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: