Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14333

ORC schema evolution from float to double changes precision and breaks filters

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.0, 2.2.0
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: ORC
    • Labels:
      None

      Description

      ORC vs text schema evolution from float to double changes precision

      Text Schema Evolution
      hive> create table float_text(f float);
      hive> insert into float_text values(74.72);
      hive> select f from float_text;
      OK
      74.72
      hive> alter table float_text change column f f double;
      hive> select f from float_text;
      OK
      74.72
      
      Orc Schema Evolution
      hive> create table float_orc(f float) stored as orc;
      hive> insert into float_orc values(74.72);
      hive> select f from float_orc;
      OK
      74.72
      hive> alter table float_orc change column f f double;
      hive> select f from float_orc;
      OK
      74.72000122070312
      

      This will break all filters on the evolved column "f"

      Filter returning no results
      hive> set hive.optimize.index.filter=false;
      hive> select f from float_orc where f=74.72;
      OK
      

        Attachments

        1. HIVE-14333.1.patch
          14 kB
          Prasanth Jayachandran
        2. HIVE-14333.2.patch
          16 kB
          Prasanth Jayachandran
        3. HIVE-14333.3.patch
          31 kB
          Prasanth Jayachandran

          Issue Links

            Activity

              People

              • Assignee:
                prasanth_j Prasanth Jayachandran
                Reporter:
                prasanth_j Prasanth Jayachandran
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: