Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8078

ORC Delta encoding corrupts data when delta overflows long

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.13.1, 0.14.0
    • 0.14.0
    • File Formats
    • None

    Description

      There is an issue with the integer encoding that can cause corruption in certain cases.
      The following 3 longs cause this failure.
      4513343538618202711
      2911390882471569739
      -9181829309989854913

      I believe that even though the numbers are in decreasing order, the delta between the last two numbers overflows causing a positive delta, in this case the last digit ends up being corrupted (the delta is applied for the wrong sign resulting in -3442132998776557225 instead of -9181829309989854913.

      Attachments

        1. HIVE-8078.1.patch
          15 kB
          Prasanth Jayachandran
        2. HIVE-8078.2.patch
          97 kB
          Prasanth Jayachandran
        3. HIVE-8078.3.patch
          97 kB
          Prasanth Jayachandran
        4. HIVE-8078.4.patch
          99 kB
          Prasanth Jayachandran
        5. HIVE-8078.5.patch
          99 kB
          Prasanth Jayachandran
        6. HIVE-8078-testcase.patch
          2 kB
          Tim Patterson

        Issue Links

          Activity

            People

              prasanth_j Prasanth Jayachandran
              tim.patterson Tim Patterson
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: