[HIVE-4409] Prevent incompatible column type changes

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.12.0
• Component/s: CLI, Metastore
• Labels: TODOC12
• Hadoop Flags: Reviewed
• Release Note: Added new config param hive.metastore.disallow.incompatible.col.type.changes which is false by default.
• Assignee: Dilip Joseph
• Reporter: Dilip Joseph
• Created: 24/Apr/13 06:04
• Updated: 03/Nov/15 06:19
• Resolved: 25/Apr/13 14:09

Description

If a user changes the type of an existing column of a partitioned table to an incompatible type, subsequent accesses of old partitions will result in a ClassCastException (see example below). We should prevent the user from making incompatible type changes. This feature will be controlled by a new config parameter.

Example:

CREATE TABLE test_table123 (a INT, b MAP<STRING, STRING>) PARTITIONED BY (ds STRING) STORED AS SEQUENCEFILE;
INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1", "b1") FROM src LIMIT 1;
SELECT * from test_table123 WHERE ds="foo1";
ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
SELECT * from test_table123 WHERE ds="foo1";

The last SELECT fails with the following exception:

Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
java.io.IOException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)

The ALTER TABLE statement is blocked if you set the following parameter, introduced int the fix to this JIRA:
SET hive.metastore.disallow.incompatible.col.type.changes=true;

Issue Links

• is depended upon by: HIVE-4424 MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409
• relates to: HIVE-12320 hive.metastore.disallow.incompatible.col.type.changes should be true by default