Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4123

The RLE encoding for ORC can be improved

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: File Formats
    • Labels:
    • Release Note:
      I just committed this. Thanks, Prasanth!
    • Tags:
      orc, rle, encoding

      Description

      The run length encoding of integers can be improved:

      • tighter bit packing
      • allow delta encoding
      • allow longer runs

        Attachments

        1. HIVE-4123.1.git.patch.txt
          194 kB
          Prasanth Jayachandran
        2. HIVE-4123.2.git.patch.txt
          140 kB
          Prasanth Jayachandran
        3. HIVE-4123.3.patch.txt
          129 kB
          Prasanth Jayachandran
        4. HIVE-4123.4.patch.txt
          118 kB
          Prasanth Jayachandran
        5. HIVE-4123.5.txt
          119 kB
          Prasanth Jayachandran
        6. HIVE-4123.6.txt
          129 kB
          Prasanth Jayachandran
        7. HIVE-4123.7.txt
          151 kB
          Prasanth Jayachandran
        8. HIVE-4123.8.txt
          150 kB
          Owen O'Malley
        9. HIVE-4123.8.txt
          150 kB
          Prasanth Jayachandran
        10. HIVE-4123.patch.txt
          150 kB
          Owen O'Malley
        11. HIVE-4123-8.patch
          150 kB
          Owen O'Malley
        12. ORC-Compression-Ratio-Comparison.xlsx
          60 kB
          Prasanth Jayachandran

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-5970 ArrayIndexOutOfBoundsException in RunLengthIntegerReaderV2.java

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-5091 ORC files should have an option to pad stripes to the HDFS block boundaries

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                prasanth_j Prasanth Jayachandran
                Reporter:
                omalley Owen O'Malley
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: