[HIVE-14333] ORC schema evolution from float to double changes precision and breaks filters

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.2.0
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: ORC
• Target Version/s: 2.1.0, 2.2.0
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 26/Jul/16 01:29
• Updated: 08/Dec/16 14:16
• Resolved: 28/Jul/16 06:51

Description

ORC vs text schema evolution from float to double changes precision

Text Schema Evolution

hive> create table float_text(f float);
hive> insert into float_text values(74.72);
hive> select f from float_text;
OK
74.72
hive> alter table float_text change column f f double;
hive> select f from float_text;
OK
74.72

Orc Schema Evolution

hive> create table float_orc(f float) stored as orc;
hive> insert into float_orc values(74.72);
hive> select f from float_orc;
OK
74.72
hive> alter table float_orc change column f f double;
hive> select f from float_orc;
OK
74.72000122070312

This will break all filters on the evolved column "f"

Filter returning no results

hive> set hive.optimize.index.filter=false;
hive> select f from float_orc where f=74.72;
OK

Issue Links

• blocks: HIVE-14310 ORC schema evolution should not completely disable PPD
• relates to: HIVE-14214 ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
• links to: RB