Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4123

The RLE encoding for ORC can be improved

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.12.0
    • 0.12.0
    • File Formats
    • I just committed this. Thanks, Prasanth!
    • orc, rle, encoding

    Description

      The run length encoding of integers can be improved:

      • tighter bit packing
      • allow delta encoding
      • allow longer runs

      Attachments

        1. ORC-Compression-Ratio-Comparison.xlsx
          60 kB
          Prasanth Jayachandran
        2. HIVE-4123-8.patch
          150 kB
          Owen O'Malley
        3. HIVE-4123.patch.txt
          150 kB
          Owen O'Malley
        4. HIVE-4123.8.txt
          150 kB
          Prasanth Jayachandran
        5. HIVE-4123.8.txt
          150 kB
          Owen O'Malley
        6. HIVE-4123.7.txt
          151 kB
          Prasanth Jayachandran
        7. HIVE-4123.6.txt
          129 kB
          Prasanth Jayachandran
        8. HIVE-4123.5.txt
          119 kB
          Prasanth Jayachandran
        9. HIVE-4123.4.patch.txt
          118 kB
          Prasanth Jayachandran
        10. HIVE-4123.3.patch.txt
          129 kB
          Prasanth Jayachandran
        11. HIVE-4123.2.git.patch.txt
          140 kB
          Prasanth Jayachandran
        12. HIVE-4123.1.git.patch.txt
          194 kB
          Prasanth Jayachandran

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-5970 ArrayIndexOutOfBoundsException in RunLengthIntegerReaderV2.java

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-5091 ORC files should have an option to pad stripes to the HDFS block boundaries

          • Major - Major loss of function.
          • Closed

          Activity

            People

              prasanth_j Prasanth Jayachandran
              omalley Owen O'Malley
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: