Hive
  1. Hive
  2. HIVE-5199

Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      The issue happens because of the changes in HIVE-3833.

      Consider a partitioned table with different custom serdes for the partition and tables. The serde at table level, say, customSerDe1's object inspector is of settableDataType where as the serde at partition level, say, customSerDe2's object inspector is of nonSettableDataType. The current implementation introduced by HIVE-3833 does not convert nested Complex Data Types which extend nonSettableObjectInspector to a settableObjectInspector type inside ObjectInspectorConverters.getConvertedOI(). However, it tries to typecast the nonSettableObjectInspector to a settableObjectInspector inside ObjectInspectorConverters.getConverter(ObjectInspector inputOI, ObjectInspector outputOI).

      The attached patch HIVE-5199.2.patch.txt contains a stand-alone test case.

      The below exception can happen via FetchOperator as well as MapOperator.
      For example, consider the FetchOperator.
      Inside FetchOperator consider the following call:
      getRecordReader()->ObjectInspectorConverters. getConverter()

      The stack trace as follows:
      2013-08-28 17:57:25,307 ERROR CliDriver (SessionState.java:printError(432)) - Failed with exception java.io.IOException:java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
      java.io.IOException: java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
      at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
      at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
      at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
      at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
      Caused by: java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:307)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:406)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)

      1. HIVE-5199.patch.txt
        6 kB
        Hari Sankar Sivarama Subramaniyan
      2. HIVE-5199.2.patch.txt
        32 kB
        Hari Sankar Sivarama Subramaniyan
      3. HIVE-5199.3.patch.txt
        26 kB
        Hari Sankar Sivarama Subramaniyan
      4. HIVE-5199.patch.4.txt
        28 kB
        Hari Sankar Sivarama Subramaniyan
      5. 0001-HIVE-5199-0.12-branch.patch
        28 kB
        Hari Sankar Sivarama Subramaniyan
      6. HIVE-5199-0.12-branch.conflict.patch
        0.9 kB
        Jason Dere

        Activity

        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -

        Adding code changes to avoid nested structures within complex datatypes to throw exception

        Show
        Hari Sankar Sivarama Subramaniyan added a comment - Adding code changes to avoid nested structures within complex datatypes to throw exception
        Hide
        Brock Noland added a comment -

        Hari,

        Thanks for the contribution! On small suggestion, in this JIRA project we discuss the Apache Hive project, not a distribution, therefore it's useful to discuss a regression in 0.9., 0.10, or 0.11 which caused this.

        Show
        Brock Noland added a comment - Hari, Thanks for the contribution! On small suggestion, in this JIRA project we discuss the Apache Hive project, not a distribution, therefore it's useful to discuss a regression in 0.9., 0.10, or 0.11 which caused this.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12601447/HIVE-5199.patch.txt

        ERROR: -1 due to 24 failed/errored test(s), 2909 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_vs_table_metadata
        org.apache.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_serde_format
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fileformat_mix
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partInit
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15
        org.apache.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/615/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/615/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 24 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12601447/HIVE-5199.patch.txt ERROR: -1 due to 24 failed/errored test(s), 2909 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_vs_table_metadata org.apache.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_serde_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fileformat_mix org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partInit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15 org.apache.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/615/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/615/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 24 tests failed This message is automatically generated.
        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -

        While testing, we need to move CustomSerDes.jar to data/files

        Show
        Hari Sankar Sivarama Subramaniyan added a comment - While testing, we need to move CustomSerDes.jar to data/files
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12601543/HIVE-5199.2.patch.txt

        ERROR: -1 due to 2 failed/errored test(s), 2910 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat17
        org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/626/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/626/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 2 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12601543/HIVE-5199.2.patch.txt ERROR: -1 due to 2 failed/errored test(s), 2910 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat17 org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/626/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/626/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12601543/HIVE-5199.2.patch.txt

        ERROR: -1 due to 6 failed/errored test(s), 2910 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat17
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead
        org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/629/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/629/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 6 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12601543/HIVE-5199.2.patch.txt ERROR: -1 due to 6 failed/errored test(s), 2910 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat17 org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/629/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/629/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 6 tests failed This message is automatically generated.
        Hide
        Ashutosh Chauhan added a comment -

        Hari Sankar Sivarama Subramaniyan Can you create a phabricator or RB link for this patch?

        Show
        Ashutosh Chauhan added a comment - Hari Sankar Sivarama Subramaniyan Can you create a phabricator or RB link for this patch?
        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -
        Show
        Hari Sankar Sivarama Subramaniyan added a comment - Created RB link : https://reviews.apache.org/r/14008/
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12601756/HIVE-5199.3.patch.txt

        ERROR: -1 due to 10 failed/errored test(s), 3085 tests executed
        Failed tests:

        org.apache.hive.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask
        org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema
        org.apache.hive.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTable
        org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTable
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead
        org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR
        org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR
        org.apache.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataNoSpec
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/648/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/648/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 10 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12601756/HIVE-5199.3.patch.txt ERROR: -1 due to 10 failed/errored test(s), 3085 tests executed Failed tests: org.apache.hive.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema org.apache.hive.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTable org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTable org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR org.apache.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataNoSpec Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/648/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/648/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 10 tests failed This message is automatically generated.
        Hide
        Ashutosh Chauhan added a comment -

        Looks good overall. Can you also add a similar test for unpartitioned table? Also test which result in MR job (select a,b from T) for both scenarios.

        Show
        Ashutosh Chauhan added a comment - Looks good overall. Can you also add a similar test for unpartitioned table? Also test which result in MR job (select a,b from T) for both scenarios.
        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -

        Uploaded test cases for cast exceptions which can happen via MapOperator as well as the FetchOperator. I ran all the failed tests from the previous iterations locally and these complete without any failures. I looked at the stack trace. The failures do not look relevant to the changes I made. Can you please confirm.
        -Hari

        Show
        Hari Sankar Sivarama Subramaniyan added a comment - Uploaded test cases for cast exceptions which can happen via MapOperator as well as the FetchOperator. I ran all the failed tests from the previous iterations locally and these complete without any failures. I looked at the stack trace. The failures do not look relevant to the changes I made. Can you please confirm. -Hari
        Hide
        Ashutosh Chauhan added a comment -

        +1

        Show
        Ashutosh Chauhan added a comment - +1
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Hari!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Hari!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #89 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/89/)
        HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155)

        • /hive/trunk/build-common.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java
        • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q
        • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out
        • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #89 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/89/ ) HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155 ) /hive/trunk/build-common.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #157 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/157/)
        HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155)

        • /hive/trunk/build-common.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java
        • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q
        • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out
        • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #157 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/157/ ) HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155 ) /hive/trunk/build-common.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -

        Ashutosh Chauhan Thanks for committing the changes. I see that data/files/pw17.txt is missing from the commit. Can you please make sure this is added to the trunk, otherwise the testcase partition_wise_fileformat17 will fail.

        Show
        Hari Sankar Sivarama Subramaniyan added a comment - Ashutosh Chauhan Thanks for committing the changes. I see that data/files/pw17.txt is missing from the commit. Can you please make sure this is added to the trunk, otherwise the testcase partition_wise_fileformat17 will fail.
        Hide
        Hari Sankar Sivarama Subramaniyan added a comment -

        Ashutosh Chauhan Please add data/files/pw17.txt to the trunk. Otherwise, partition_wise_fileformat17 will fail.
        Hari

        Show
        Hari Sankar Sivarama Subramaniyan added a comment - Ashutosh Chauhan Please add data/files/pw17.txt to the trunk. Otherwise, partition_wise_fileformat17 will fail. Hari
        Hide
        Ashutosh Chauhan added a comment -

        Fixed via HIVE-5249

        Show
        Ashutosh Chauhan added a comment - Fixed via HIVE-5249
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-h0.21 #2322 (See https://builds.apache.org/job/Hive-trunk-h0.21/2322/)
        HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155)

        • /hive/trunk/build-common.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java
        • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q
        • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out
        • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2322 (See https://builds.apache.org/job/Hive-trunk-h0.21/2322/ ) HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155 ) /hive/trunk/build-common.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Hide
        Hudson added a comment -

        ABORTED: Integrated in Hive-trunk-hadoop2 #419 (See https://builds.apache.org/job/Hive-trunk-hadoop2/419/)
        HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155)

        • /hive/trunk/build-common.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java
        • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q
        • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out
        • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Show
        Hudson added a comment - ABORTED: Integrated in Hive-trunk-hadoop2 #419 (See https://builds.apache.org/job/Hive-trunk-hadoop2/419/ ) HIVE-5199 : Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 (Hari Sankar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521155 ) /hive/trunk/build-common.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableListObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomNonSettableStructObjectInspector1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe1.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/serde2/CustomSerDe2.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat17.q /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat17.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
        Hide
        Thejas M Nair added a comment -

        Patch committed to 0.12 branch.

        Show
        Thejas M Nair added a comment - Patch committed to 0.12 branch.
        Hide
        Jason Dere added a comment -

        Applying these changes in the 0.12 branch stomped some changes in HIVE-5206 since they touch some of the same code. Attaching a fix for the conflict.

        Show
        Jason Dere added a comment - Applying these changes in the 0.12 branch stomped some changes in HIVE-5206 since they touch some of the same code. Attaching a fix for the conflict.
        Hide
        Thejas M Nair added a comment -

        Thanks Jason! Committed the conflict fix to 0.12 branch.

        Show
        Thejas M Nair added a comment - Thanks Jason! Committed the conflict fix to 0.12 branch.
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          People

          • Assignee:
            Hari Sankar Sivarama Subramaniyan
            Reporter:
            Hari Sankar Sivarama Subramaniyan
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development