Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17530

ClassCastException when converting uniontype

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0, 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      To repro:

      SET hive.exec.schema.evolution = false;
      
      CREATE TABLE avro_orc_partitioned_uniontype (a uniontype<boolean, string>) PARTITIONED BY (b int) STORED AS ORC;
      
      INSERT INTO avro_orc_partitioned_uniontype PARTITION (b=1) SELECT create_union(1, true, value) FROM src LIMIT 5;
      
      ALTER TABLE avro_orc_partitioned_uniontype SET FILEFORMAT AVRO;
      
      SELECT * FROM avro_orc_partitioned_uniontype;
      

      The exception you get is:

      java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.UnionObject
      

      The issue is that StandardUnionObjectInspector was creating and returning an ArrayList rather than a UnionObject.

      1. HIVE-17530.1.patch
        7 kB
        Anthony Hsu
      2. HIVE-17530.2.patch
        11 kB
        Anthony Hsu

        Issue Links

          Activity

          Hide
          erwaman Anthony Hsu added a comment -

          Attached patch. Also posted RB: https://reviews.apache.org/r/62321/

          Show
          erwaman Anthony Hsu added a comment - Attached patch. Also posted RB: https://reviews.apache.org/r/62321/
          Hide
          cwsteinbach Carl Steinbach added a comment -

          Patch looks good. +1. Will commit if tests pass.

          Show
          cwsteinbach Carl Steinbach added a comment - Patch looks good. +1. Will commit if tests pass.
          Hide
          erwaman Anthony Hsu added a comment -

          Uploaded new patch. Changes:

          • Fixed test TestObjectInspectorConverters.testObjectInspectorConverters()
          • Renamed SettableUnionObjectInspector.addField() to setFieldAndTag().

          Also updated RB: https://reviews.apache.org/r/62321/

          Show
          erwaman Anthony Hsu added a comment - Uploaded new patch. Changes: Fixed test TestObjectInspectorConverters.testObjectInspectorConverters() Renamed SettableUnionObjectInspector.addField() to setFieldAndTag(). Also updated RB: https://reviews.apache.org/r/62321/
          Hide
          rdsr Ratandeep Ratti added a comment -
          Show
          rdsr Ratandeep Ratti added a comment - LGTM Anthony Hsu
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12887239/HIVE-17530.2.patch

          SUCCESS: +1 due to 2 test(s) being added or modified.

          ERROR: -1 due to 11 failed/errored test(s), 11041 tests executed
          Failed tests:

          TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
          TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=156)
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2] (batchId=89)
          org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234)
          org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=234)
          org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
          org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=215)
          org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testStoreFuncAllSimpleTypes (batchId=183)
          org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteTimestamp (batchId=183)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6830/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6830/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6830/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 11 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12887239 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12887239/HIVE-17530.2.patch SUCCESS: +1 due to 2 test(s) being added or modified. ERROR: -1 due to 11 failed/errored test(s), 11041 tests executed Failed tests: TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=156) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2] (batchId=89) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=234) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=215) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testStoreFuncAllSimpleTypes (batchId=183) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteTimestamp (batchId=183) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6830/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6830/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6830/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 11 tests failed This message is automatically generated. ATTACHMENT ID: 12887239 - PreCommit-HIVE-Build
          Hide
          cwsteinbach Carl Steinbach added a comment -

          Committed to master. Thanks Anthony!

          Show
          cwsteinbach Carl Steinbach added a comment - Committed to master. Thanks Anthony!
          Hide
          erwaman Anthony Hsu added a comment -
          Show
          erwaman Anthony Hsu added a comment - Thanks, Carl Steinbach and Ratandeep Ratti !

            People

            • Assignee:
              erwaman Anthony Hsu
              Reporter:
              erwaman Anthony Hsu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development