Hive
  1. Hive
  2. HIVE-4409

Prevent incompatible column type changes

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.12.0
    • Component/s: CLI, Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Added new config param hive.metastore.disallow.incompatible.col.type.changes which is false by default.

      Description

      If a user changes the type of an existing column of a partitioned table to an incompatible type, subsequent accesses of old partitions will result in a ClassCastException (see example below). We should prevent the user from making incompatible type changes. This feature will be controlled by a new config parameter.

      Example:

      CREATE TABLE test_table123 (a INT, b MAP<STRING, STRING>) PARTITIONED BY (ds STRING) STORED AS SEQUENCEFILE;
      INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1", "b1") FROM src LIMIT 1;
      SELECT * from test_table123 WHERE ds="foo1";
      ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
      SELECT * from test_table123 WHERE ds="foo1";

      The last SELECT fails with the following exception:

      Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
      java.io.IOException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
      at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
      at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
      at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
      at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
      at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)

      The ALTER TABLE statement is blocked if you set the following parameter, introduced int the fix to this JIRA:
      SET hive.metastore.disallow.incompatible.col.type.changes=true;

      1. HIVE-4409.D10539.1.patch
        20 kB
        Phabricator
      2. HIVE-4409.D10539.2.patch
        21 kB
        Phabricator
      3. hive.4409.1.patch
        22 kB
        Namit Jain

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #176 (See https://builds.apache.org/job/Hive-trunk-hadoop2/176/)
          HIVE-4424 MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409
          (Namit Jain) (Revision 1476119)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476119
          Files :

          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #176 (See https://builds.apache.org/job/Hive-trunk-hadoop2/176/ ) HIVE-4424 MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409 (Namit Jain) (Revision 1476119) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476119 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #2079 (See https://builds.apache.org/job/Hive-trunk-h0.21/2079/)
          HIVE-4424 MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409
          (Namit Jain) (Revision 1476119)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476119
          Files :

          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #2079 (See https://builds.apache.org/job/Hive-trunk-h0.21/2079/ ) HIVE-4424 MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409 (Namit Jain) (Revision 1476119) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476119 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #175 (See https://builds.apache.org/job/Hive-trunk-hadoop2/175/)
          HIVE-4409 Prevent incompatible column type changes
          (Dilip Joseph via namit) (Revision 1475783)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1475783
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          • /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q
          • /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q
          • /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out
          • /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #175 (See https://builds.apache.org/job/Hive-trunk-hadoop2/175/ ) HIVE-4409 Prevent incompatible column type changes (Dilip Joseph via namit) (Revision 1475783) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1475783 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q /hive/trunk/ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out /hive/trunk/ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out
          Hide
          Namit Jain added a comment -

          my bad, will do

          Show
          Namit Jain added a comment - my bad, will do
          Hide
          Amareshwari Sriramadasu added a comment -

          Looks like the commit checked in /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig as well.

          Namit Jain, Do you want to remove it?

          Show
          Amareshwari Sriramadasu added a comment - Looks like the commit checked in /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig as well. Namit Jain , Do you want to remove it?
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #2078 (See https://builds.apache.org/job/Hive-trunk-h0.21/2078/)
          HIVE-4409 Prevent incompatible column type changes
          (Dilip Joseph via namit) (Revision 1475783)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1475783
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig
          • /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q
          • /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q
          • /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out
          • /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #2078 (See https://builds.apache.org/job/Hive-trunk-h0.21/2078/ ) HIVE-4409 Prevent incompatible column type changes (Dilip Joseph via namit) (Revision 1475783) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1475783 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java.orig /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q /hive/trunk/ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q /hive/trunk/ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out /hive/trunk/ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out /hive/trunk/ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out
          Hide
          Namit Jain added a comment -

          Committed. Thanks Dilip

          Show
          Namit Jain added a comment - Committed. Thanks Dilip
          Hide
          Namit Jain added a comment -

          +1

          Show
          Namit Jain added a comment - +1
          Hide
          Phabricator added a comment -

          njain has accepted the revision "HIVE-4409 [jira] Prevent incompatible column type changes".

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          BRANCH
          trunk

          ARCANIST PROJECT
          hive

          To: JIRA, njain, kevinwilfong, dilipj
          Cc: gangtimliu, sambavim

          Show
          Phabricator added a comment - njain has accepted the revision " HIVE-4409 [jira] Prevent incompatible column type changes". REVISION DETAIL https://reviews.facebook.net/D10539 BRANCH trunk ARCANIST PROJECT hive To: JIRA, njain, kevinwilfong, dilipj Cc: gangtimliu, sambavim
          Hide
          Phabricator added a comment -

          dilipj has commented on the revision "HIVE-4409 [jira] Prevent incompatible column type changes".

          INLINE COMMENTS
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:374 Done
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:344 Done

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          To: JIRA, njain, kevinwilfong, dilipj
          Cc: gangtimliu, sambavim

          Show
          Phabricator added a comment - dilipj has commented on the revision " HIVE-4409 [jira] Prevent incompatible column type changes". INLINE COMMENTS metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:374 Done common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:344 Done REVISION DETAIL https://reviews.facebook.net/D10539 To: JIRA, njain, kevinwilfong, dilipj Cc: gangtimliu, sambavim
          Hide
          Phabricator added a comment -

          dilipj updated the revision "HIVE-4409 [jira] Prevent incompatible column type changes".

          Added comment's as per Namit's review

          Reviewers: JIRA, njain, kevinwilfong

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          CHANGE SINCE LAST DIFF
          https://reviews.facebook.net/D10539?vs=32883&id=32895#toc

          AFFECTED FILES
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          conf/hive-default.xml.template
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q
          ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q
          ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q
          ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out
          ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out
          ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out

          To: JIRA, njain, kevinwilfong, dilipj
          Cc: gangtimliu, sambavim

          Show
          Phabricator added a comment - dilipj updated the revision " HIVE-4409 [jira] Prevent incompatible column type changes". Added comment's as per Namit's review Reviewers: JIRA, njain, kevinwilfong REVISION DETAIL https://reviews.facebook.net/D10539 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10539?vs=32883&id=32895#toc AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java conf/hive-default.xml.template metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out To: JIRA, njain, kevinwilfong, dilipj Cc: gangtimliu, sambavim
          Hide
          Phabricator added a comment -

          njain has commented on the revision "HIVE-4409 [jira] Prevent incompatible column type changes".

          INLINE COMMENTS
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:344 can you explain this new parameter in hive-default.xml.template ?
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:374 Can you add a big comment here ?

          RCFile default serde (columnarSerde) serializes the values in such a way that the
          datatypes can be converted from string to any type. The map is also serialized as
          a string, which can be read as a string as well. However, with any binary serialization,
          this is not true.


          Add the same comment in hive-default.xml.template also where you are explaining this
          new parameter.

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          To: JIRA, njain, kevinwilfong, dilipj
          Cc: gangtimliu, sambavim

          Show
          Phabricator added a comment - njain has commented on the revision " HIVE-4409 [jira] Prevent incompatible column type changes". INLINE COMMENTS common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:344 can you explain this new parameter in hive-default.xml.template ? metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:374 Can you add a big comment here ? RCFile default serde (columnarSerde) serializes the values in such a way that the datatypes can be converted from string to any type. The map is also serialized as a string, which can be read as a string as well. However, with any binary serialization, this is not true. – Add the same comment in hive-default.xml.template also where you are explaining this new parameter. REVISION DETAIL https://reviews.facebook.net/D10539 To: JIRA, njain, kevinwilfong, dilipj Cc: gangtimliu, sambavim
          Hide
          Namit Jain added a comment -

          comments

          Show
          Namit Jain added a comment - comments
          Hide
          Phabricator added a comment -

          dilipj has added CCs to the revision "HIVE-4409 [jira] Prevent incompatible column type changes".
          Added CCs: gangtimliu, sambavim

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          To: JIRA, njain, kevinwilfong, dilipj
          Cc: gangtimliu, sambavim

          Show
          Phabricator added a comment - dilipj has added CCs to the revision " HIVE-4409 [jira] Prevent incompatible column type changes". Added CCs: gangtimliu, sambavim REVISION DETAIL https://reviews.facebook.net/D10539 To: JIRA, njain, kevinwilfong, dilipj Cc: gangtimliu, sambavim
          Hide
          Phabricator added a comment -

          dilipj has added reviewers to the revision "HIVE-4409 [jira] Prevent incompatible column type changes".
          Added Reviewers: njain, kevinwilfong

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          To: JIRA, njain, kevinwilfong, dilipj

          Show
          Phabricator added a comment - dilipj has added reviewers to the revision " HIVE-4409 [jira] Prevent incompatible column type changes". Added Reviewers: njain, kevinwilfong REVISION DETAIL https://reviews.facebook.net/D10539 To: JIRA, njain, kevinwilfong, dilipj
          Hide
          Phabricator added a comment -

          dilipj requested code review of "HIVE-4409 [jira] Prevent incompatible column type changes".

          Reviewers: JIRA

          Disable incompatible column type changes

          JIRA: HIVE-4409

          If a user changes the type of an existing column of a partitioned table to an incompatible type, subsequent accesses of old partitions will result in a ClassCastException (see example below). We should prevent the user from making incompatible type changes. This feature will be controlled by a new config parameter.

          Example:

          CREATE TABLE test_table123 (a INT, b MAP<STRING, STRING>) PARTITIONED BY (ds STRING) STORED AS SEQUENCEFILE;
          INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1", "b1") FROM src LIMIT 1;
          SELECT * from test_table123 WHERE ds="foo1";
          SET hive.metastore.disallow.invalid.col.type.changes=true;
          ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING);
          SELECT * from test_table123 WHERE ds="foo1";

          The last SELECT fails with the following exception:

          Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
          java.io.IOException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
          at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
          at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
          at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
          at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406)
          at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
          at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
          at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
          at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
          at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
          at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
          at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108)

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D10539

          AFFECTED FILES
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q
          ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q
          ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q
          ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out
          ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out
          ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out

          MANAGE HERALD RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/25215/

          To: JIRA, dilipj

          Show
          Phabricator added a comment - dilipj requested code review of " HIVE-4409 [jira] Prevent incompatible column type changes". Reviewers: JIRA Disable incompatible column type changes JIRA: HIVE-4409 If a user changes the type of an existing column of a partitioned table to an incompatible type, subsequent accesses of old partitions will result in a ClassCastException (see example below). We should prevent the user from making incompatible type changes. This feature will be controlled by a new config parameter. Example: CREATE TABLE test_table123 (a INT, b MAP<STRING, STRING>) PARTITIONED BY (ds STRING) STORED AS SEQUENCEFILE; INSERT OVERWRITE TABLE test_table123 PARTITION(ds="foo1") SELECT 1, MAP("a1", "b1") FROM src LIMIT 1; SELECT * from test_table123 WHERE ds="foo1"; SET hive.metastore.disallow.invalid.col.type.changes=true; ALTER TABLE test_table123 REPLACE COLUMNS (a INT, b STRING); SELECT * from test_table123 WHERE ds="foo1"; The last SELECT fails with the following exception: Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector java.io.IOException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1406) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790) at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_class_cast(TestCliDriver.java:108) TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D10539 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on1.q ql/src/test/queries/clientnegative/disallow_incompatible_type_change_on2.q ql/src/test/queries/clientpositive/disallow_incompatible_type_change_off.q ql/src/test/results/clientnegative/disallow_incompatible_type_change_on1.q.out ql/src/test/results/clientnegative/disallow_incompatible_type_change_on2.q.out ql/src/test/results/clientpositive/disallow_incompatible_type_change_off.q.out MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/25215/ To: JIRA, dilipj

            People

            • Assignee:
              Dilip Joseph
              Reporter:
              Dilip Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development