Hive
  1. Hive
  2. HIVE-3833

object inspectors should be initialized based on partition metadata

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Query Processor
    • Labels:
      None
    • Release Note:
      Rows in partitions are now read using partition schema and than made to comply with table schema, instead of being read directly using table schema.

      Description

      Currently, different partitions can be picked up for the same input split based on the
      serdes' etc. And, we dont allow to change the schema for LazyColumnarBinarySerDe.
      Instead of that, different partitions should be part of the same split, only if the
      partition schemas exactly match. The operator tree object inspectors should be based
      on the partition schema. That would give greater flexibility and also help using binary serde with rcfile

      1. hive.3833.9.patch
        103 kB
        Namit Jain
      2. hive.3833.8.patch
        108 kB
        Namit Jain
      3. hive.3833.7.patch
        583 kB
        Namit Jain
      4. hive.3833.6.patch
        583 kB
        Namit Jain
      5. hive.3833.5.patch
        573 kB
        Namit Jain
      6. hive.3833.4.patch
        97 kB
        Namit Jain
      7. hive.3833.3.patch
        74 kB
        Namit Jain
      8. hive.3833.23.patch
        301 kB
        Namit Jain
      9. hive.3833.22.patch
        275 kB
        Namit Jain
      10. hive.3833.21.patch
        274 kB
        Namit Jain
      11. hive.3833.20.patch
        273 kB
        Namit Jain
      12. hive.3833.2.patch
        74 kB
        Namit Jain
      13. hive.3833.19.patch
        273 kB
        Namit Jain
      14. hive.3833.18.patch
        273 kB
        Namit Jain
      15. hive.3833.17.patch
        265 kB
        Namit Jain
      16. hive.3833.16.path
        164 kB
        Namit Jain
      17. hive.3833.14.patch
        265 kB
        Namit Jain
      18. hive.3833.13.patch
        278 kB
        Namit Jain
      19. hive.3833.12.patch
        281 kB
        Namit Jain
      20. hive.3833.11.patch
        280 kB
        Namit Jain
      21. hive.3833.10.patch
        207 kB
        Namit Jain
      22. hive.3833.1.patch
        26 kB
        Namit Jain

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          12d 20h 10m 1 Namit Jain 14/Jan/13 10:59
          Open Open Patch Available Patch Available
          9d 15h 17m 2 Namit Jain 14/Jan/13 16:04
          Patch Available Patch Available Resolved Resolved
          10d 2h 8m 1 Ashutosh Chauhan 24/Jan/13 18:12
          Resolved Resolved Closed Closed
          112d 2h 57m 1 Owen O'Malley 16/May/13 22:10
          Mariano Dominguez made changes -
          Link This issue relates to HIVE-7046 [ HIVE-7046 ]
          Owen O'Malley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Namit Jain added a comment -

          Jakob Homan, this was definitely not intentional. Unfortunately, there was no test case, so I missed this.
          Can you provide me a complete testcase ? I will take a look.

          Show
          Namit Jain added a comment - Jakob Homan , this was definitely not intentional. Unfortunately, there was no test case, so I missed this. Can you provide me a complete testcase ? I will take a look.
          Hide
          Jakob Homan added a comment -

          This patch has broken Avro (and probably HBase and Cassandra) for partitioned tables since it no longer passes the table properties down to the serde:

          +    Properties partProps =
          +        (pd.getPartSpec() == null || pd.getPartSpec().isEmpty()) ?
          +            pd.getTableDesc().getProperties() : pd.getProperties();

          Was this intentional? If so, it's a breaking change and should be marked as such. If not, since it's not been in a release yet, can we revert the patch? See HIVE-3953.

          Show
          Jakob Homan added a comment - This patch has broken Avro (and probably HBase and Cassandra) for partitioned tables since it no longer passes the table properties down to the serde: + Properties partProps = + (pd.getPartSpec() == null || pd.getPartSpec().isEmpty()) ? + pd.getTableDesc().getProperties() : pd.getProperties(); Was this intentional? If so, it's a breaking change and should be marked as such. If not, since it's not been in a release yet, can we revert the patch? See HIVE-3953 .
          Mark Wagner made changes -
          Link This issue relates to HIVE-3953 [ HIVE-3953 ]
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #86 (See https://builds.apache.org/job/Hive-trunk-hadoop2/86/)
          HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111)

          Result = FAILURE
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input42.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out
          • /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #86 (See https://builds.apache.org/job/Hive-trunk-hadoop2/86/ ) HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out /hive/trunk/ql/src/test/results/clientpositive/input23.q.out /hive/trunk/ql/src/test/results/clientpositive/input42.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out /hive/trunk/ql/src/test/results/clientpositive/join26.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join9.q.out /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out /hive/trunk/ql/src/test/results/clientpositive/union22.q.out /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1935 (See https://builds.apache.org/job/Hive-trunk-h0.21/1935/)
          HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111)

          Result = SUCCESS
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input42.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out
          • /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1935 (See https://builds.apache.org/job/Hive-trunk-h0.21/1935/ ) HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out /hive/trunk/ql/src/test/results/clientpositive/input23.q.out /hive/trunk/ql/src/test/results/clientpositive/input42.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out /hive/trunk/ql/src/test/results/clientpositive/join26.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join9.q.out /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out /hive/trunk/ql/src/test/results/clientpositive/union22.q.out /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Hide
          Hudson added a comment -

          Integrated in hive-trunk-hadoop1 #41 (See https://builds.apache.org/job/hive-trunk-hadoop1/41/)
          HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q
          • /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input42.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out
          • /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml
          • /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Show
          Hudson added a comment - Integrated in hive-trunk-hadoop1 #41 (See https://builds.apache.org/job/hive-trunk-hadoop1/41/ ) HIVE-3833 : object inspectors should be initialized based on partition metadata (Namit Jain via Ashutosh Chauhan) (Revision 1438111) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438111 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/util/ObjectPair.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat10.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat11.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat12.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat13.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat14.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat8.q /hive/trunk/ql/src/test/queries/clientpositive/partition_wise_fileformat9.q /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_4.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_6.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketcontext_8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin10.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin11.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin12.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin13.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin5.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin8.q.out /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out /hive/trunk/ql/src/test/results/clientpositive/combine2_hadoop20.q.out /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out /hive/trunk/ql/src/test/results/clientpositive/groupby_sort_6.q.out /hive/trunk/ql/src/test/results/clientpositive/input23.q.out /hive/trunk/ql/src/test/results/clientpositive/input42.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out /hive/trunk/ql/src/test/results/clientpositive/input_part9.q.out /hive/trunk/ql/src/test/results/clientpositive/join26.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join9.q.out /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part8.q.out /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat10.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat11.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat12.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat13.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat14.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat8.q.out /hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat9.q.out /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out /hive/trunk/ql/src/test/results/clientpositive/ppd_union_view.q.out /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner2.q.out /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out /hive/trunk/ql/src/test/results/clientpositive/sample1.q.out /hive/trunk/ql/src/test/results/clientpositive/sample10.q.out /hive/trunk/ql/src/test/results/clientpositive/sample8.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_11.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_12.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out /hive/trunk/ql/src/test/results/clientpositive/union22.q.out /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/NullStructSerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableStructObjectInspector.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StructObjectInspector.java
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note Rows in partitions are now read using partition schema and than made to comply with table schema, instead of being read directly using table schema.
          Fix Version/s 0.11.0 [ 12323587 ]
          Resolution Fixed [ 1 ]
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Namit!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Namit!
          Hide
          Namit Jain added a comment -

          Yes, the tests passed for me for .23

          Show
          Namit Jain added a comment - Yes, the tests passed for me for .23
          Hide
          Ashutosh Chauhan added a comment -

          +1 for latest patch. Is .23 the latest complete patch ? Running tests on that now.

          Show
          Ashutosh Chauhan added a comment - +1 for latest patch. Is .23 the latest complete patch ? Running tests on that now.
          Namit Jain made changes -
          Attachment hive.3833.23.patch [ 12566109 ]
          Namit Jain made changes -
          Attachment hive.3833.23.patch [ 12566108 ]
          Namit Jain made changes -
          Attachment hive.3833.23.patch [ 12566108 ]
          Namit Jain made changes -
          Attachment hive.3833.22.patch [ 12566098 ]
          Namit Jain made changes -
          Attachment hive.3833.22.patch [ 12566092 ]
          Namit Jain made changes -
          Attachment hive.3833.22.patch [ 12566092 ]
          Namit Jain made changes -
          Attachment hive.3833.21.patch [ 12566088 ]
          Hide
          Ashutosh Chauhan added a comment -

          For the above, it is fairly difficult to address. In a follow-up, I can add a serde level property, which indicates that the serde can handle different datatypes (for eg. lazySimpleSerde) - if all the partitions of the table have serde's with this property, then we can use identityConverter. This is kind of hacky, and am not sure if it is useful, since it should not be a common case. Usually, the partition schema should match the table schema.

          I think this really is a common case. Folks usually change the serde of an existing table usually when they find a better FileFormat or sometime when there is a better serde, both of which is a rare occurrence. So, I think we need to think about optimizing this case. Though I agree approach you suggested is hacky. We need to think of a better approach, probably in a follow-up jira.

          Also thanks for updating the patch. Some more comments on latest patch are on phabricator. Also are we going to loose any lazy aspects of deserialization here? I guess not, because we are just wiring up OIs. But, want to make sure. Can you verify?

          Show
          Ashutosh Chauhan added a comment - For the above, it is fairly difficult to address. In a follow-up, I can add a serde level property, which indicates that the serde can handle different datatypes (for eg. lazySimpleSerde) - if all the partitions of the table have serde's with this property, then we can use identityConverter. This is kind of hacky, and am not sure if it is useful, since it should not be a common case. Usually, the partition schema should match the table schema. I think this really is a common case. Folks usually change the serde of an existing table usually when they find a better FileFormat or sometime when there is a better serde, both of which is a rare occurrence. So, I think we need to think about optimizing this case. Though I agree approach you suggested is hacky. We need to think of a better approach, probably in a follow-up jira. Also thanks for updating the patch. Some more comments on latest patch are on phabricator. Also are we going to loose any lazy aspects of deserialization here? I guess not, because we are just wiring up OIs. But, want to make sure. Can you verify?
          Hide
          Namit Jain added a comment -

          The tests finished fine.

          Show
          Namit Jain added a comment - The tests finished fine.
          Hide
          Namit Jain added a comment -

          In case of identity converter, there is no conversion cost, but in case of non-identity this will be worse than current impl, since converter will examine every single column value, which wasn't the case earlier. However, it's not clear how expensive this would be?

          For the above, it is fairly difficult to address. In a follow-up, I can add a serde level property, which indicates that the serde can handle different datatypes (for eg.
          lazySimpleSerde) - if all the partitions of the table have serde's with this property, then we can use identityConverter. This is kind of hacky, and am not sure if it is
          useful, since it should not be a common case. Usually, the partition schema should match the table schema.

          Show
          Namit Jain added a comment - In case of identity converter, there is no conversion cost, but in case of non-identity this will be worse than current impl, since converter will examine every single column value, which wasn't the case earlier. However, it's not clear how expensive this would be? For the above, it is fairly difficult to address. In a follow-up, I can add a serde level property, which indicates that the serde can handle different datatypes (for eg. lazySimpleSerde) - if all the partitions of the table have serde's with this property, then we can use identityConverter. This is kind of hacky, and am not sure if it is useful, since it should not be a common case. Usually, the partition schema should match the table schema.
          Hide
          Namit Jain added a comment -

          Addressed the comments including the last one.

          Show
          Namit Jain added a comment - Addressed the comments including the last one.
          Namit Jain made changes -
          Attachment hive.3833.20.patch [ 12565935 ]
          Namit Jain made changes -
          Attachment hive.3833.19.patch [ 12565932 ]
          Namit Jain made changes -
          Attachment hive.3833.18.patch [ 12565931 ]
          Hide
          Ashutosh Chauhan added a comment -

          Comments on phabricator.

          • I have made bunch of requests to rename functions, feel free to use better names than what I suggested if you feel like.
          • I have not reviewed the new tests that you have added. I assume you have verified those.
          • pm.retrieveAll() change in ObjectStore() is of concern to me. If the comments I made there are valid, please take time to see if we can do something better there.
          Show
          Ashutosh Chauhan added a comment - Comments on phabricator. I have made bunch of requests to rename functions, feel free to use better names than what I suggested if you feel like. I have not reviewed the new tests that you have added. I assume you have verified those. pm.retrieveAll() change in ObjectStore() is of concern to me. If the comments I made there are valid, please take time to see if we can do something better there.
          Hide
          Namit Jain added a comment -

          Refreshed, tests passed.

          Show
          Namit Jain added a comment - Refreshed, tests passed.
          Namit Jain made changes -
          Attachment hive.3833.17.patch [ 12565795 ]
          Namit Jain made changes -
          Attachment hive.3833.16.path [ 12565758 ]
          Hide
          Namit Jain added a comment -

          Only if the 2 schemas are different, otherwise it is identityConverter

          Show
          Namit Jain added a comment - Only if the 2 schemas are different, otherwise it is identityConverter
          Hide
          Ashutosh Chauhan added a comment -

          Could this possibly result in performance hit (CPU)? Earlier, data was deserialized per table schema, now it will be first deserialized per partition schema and than converted to comply with table schema.

          Show
          Ashutosh Chauhan added a comment - Could this possibly result in performance hit (CPU)? Earlier, data was deserialized per table schema, now it will be first deserialized per partition schema and than converted to comply with table schema.
          Hide
          Namit Jain added a comment -

          Ashutosh Chauhan, I am not refreshing, so some of the test results may need to be updated.
          Refreshing should not lead to major code changes, so you can still review the code changes.

          Show
          Namit Jain added a comment - Ashutosh Chauhan , I am not refreshing, so some of the test results may need to be updated. Refreshing should not lead to major code changes, so you can still review the code changes.
          Hide
          Namit Jain added a comment -

          Today, pathToPartitionInfo actually contains path -> TableInfo
          numPartitions is part of TableInfo.

          Since, I have changed it to be path -> partitionInfo, numPartitions is going away.

          Show
          Namit Jain added a comment - Today, pathToPartitionInfo actually contains path -> TableInfo numPartitions is part of TableInfo. Since, I have changed it to be path -> partitionInfo, numPartitions is going away.
          Hide
          Ashutosh Chauhan added a comment -

          Thanks for explaining. Makes sense. I will take a closer look at this tomorrow. Briefly looking at diffs, from many .q.out files numPartitions is getting removed. That looks like loss of info. Whats the reason for that?

          Show
          Ashutosh Chauhan added a comment - Thanks for explaining. Makes sense. I will take a closer look at this tomorrow. Briefly looking at diffs, from many .q.out files numPartitions is getting removed. That looks like loss of info. Whats the reason for that?
          Hide
          Namit Jain added a comment -

          Seems to me, this patch will take away the flexibility of combining partitions of different schemas in one split. That sounds like lesser flexibility instead of more.

          No, I am not sure whether I added a test for that, but that should be possible. We know when a partition is being changed.

          Shouldn't we be fixing LazyColumnarBinarySerde in that case, instead of restricting combining of partitions of different schemas in one split?

          That is not the problem (combining partitions) - the problem is that any binary serde will use the datatypes for serialization, i.e it will have different storage for int and string - otherwise, what is the point of it being binary ? In case case, unless we use the partition schema (instead of
          table schema), we can get wrong results.

          Show
          Namit Jain added a comment - Seems to me, this patch will take away the flexibility of combining partitions of different schemas in one split. That sounds like lesser flexibility instead of more. No, I am not sure whether I added a test for that, but that should be possible. We know when a partition is being changed. Shouldn't we be fixing LazyColumnarBinarySerde in that case, instead of restricting combining of partitions of different schemas in one split? That is not the problem (combining partitions) - the problem is that any binary serde will use the datatypes for serialization, i.e it will have different storage for int and string - otherwise, what is the point of it being binary ? In case case, unless we use the partition schema (instead of table schema), we can get wrong results.
          Hide
          Ashutosh Chauhan added a comment -

          Instead of that, different partitions should be part of the same split, only if the partition schemas exactly match. That would give greater flexibility

          Seems to me, this patch will take away the flexibility of combining partitions of different schemas in one split. That sounds like lesser flexibility instead of more.

          And, we dont allow to change the schema for LazyColumnarBinarySerDe.

          Shouldn't we be fixing LazyColumnarBinarySerde in that case, instead of restricting combining of partitions of different schemas in one split?

          Show
          Ashutosh Chauhan added a comment - Instead of that, different partitions should be part of the same split, only if the partition schemas exactly match. That would give greater flexibility Seems to me, this patch will take away the flexibility of combining partitions of different schemas in one split. That sounds like lesser flexibility instead of more. And, we dont allow to change the schema for LazyColumnarBinarySerDe. Shouldn't we be fixing LazyColumnarBinarySerde in that case, instead of restricting combining of partitions of different schemas in one split?
          Namit Jain made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Namit Jain added a comment -

          Refreshed, all the tests passed.

          https://issues.apache.org/jira/secure/attachment/12564718/hive.3833.14.patch contains all the changes.

          The phabricatr entry does not contain the changes for test results compiler files, since it was exceeding the limit.

          Show
          Namit Jain added a comment - Refreshed, all the tests passed. https://issues.apache.org/jira/secure/attachment/12564718/hive.3833.14.patch contains all the changes. The phabricatr entry does not contain the changes for test results compiler files, since it was exceeding the limit.
          Namit Jain made changes -
          Attachment hive.3833.14.patch [ 12564718 ]
          Namit Jain made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Namit Jain added a comment -

          refreshing

          Show
          Namit Jain added a comment - refreshing
          Namit Jain made changes -
          Attachment hive.3833.13.patch [ 12564684 ]
          Namit Jain made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Namit Jain added a comment -

          The patch hive.3833.12.patch is the complete patch.

          The phabricator entry does not contain the changes for test results compiler files.

          Show
          Namit Jain added a comment - The patch hive.3833.12.patch is the complete patch. The phabricator entry does not contain the changes for test results compiler files.
          Namit Jain made changes -
          Attachment hive.3833.12.patch [ 12562845 ]
          Namit Jain made changes -
          Attachment hive.3833.11.patch [ 12562843 ]
          Namit Jain made changes -
          Attachment hive.3833.11.patch [ 12562842 ]
          Namit Jain made changes -
          Attachment hive.3833.11.patch [ 12562842 ]
          Namit Jain made changes -
          Attachment hive.3833.10.patch [ 12562783 ]
          Namit Jain made changes -
          Attachment hive.3833.9.patch [ 12562779 ]
          Namit Jain made changes -
          Attachment hive.3833.8.patch [ 12562778 ]
          Namit Jain made changes -
          Attachment hive.3833.7.patch [ 12562770 ]
          Namit Jain made changes -
          Attachment hive.3833.7.patch [ 12562767 ]
          Namit Jain made changes -
          Attachment hive.3833.7.patch [ 12562767 ]
          Namit Jain made changes -
          Attachment hive.3833.6.patch [ 12562762 ]
          Namit Jain made changes -
          Attachment hive.3833.5.patch [ 12562703 ]
          Namit Jain made changes -
          Attachment hive.3833.4.patch [ 12562577 ]
          Hide
          Namit Jain added a comment -

          Running tests now.

          The basic idea is: use partition metadata to read the data. Convert it to use table metadata, the rest of the
          stack does not need to know about the conversion.

          Show
          Namit Jain added a comment - Running tests now. The basic idea is: use partition metadata to read the data. Convert it to use table metadata, the rest of the stack does not need to know about the conversion.
          Namit Jain made changes -
          Attachment hive.3833.3.patch [ 12562570 ]
          Namit Jain made changes -
          Attachment hive.3833.2.patch [ 12562568 ]
          Namit Jain made changes -
          Attachment hive.3833.1.patch [ 12562453 ]
          Show
          Namit Jain added a comment - https://reviews.facebook.net/D7653
          Hide
          Namit Jain added a comment -

          The object inspectors need to be initialized based on partition metadata.
          That leaves us with the following options:
          1. Create an operator tree per partition
          2. Create a dummy operator after table scan (which converts the partition data into table data).
          This operator will be different for different inputs.

          Option 2. seems like a better option.

          Show
          Namit Jain added a comment - The object inspectors need to be initialized based on partition metadata. That leaves us with the following options: 1. Create an operator tree per partition 2. Create a dummy operator after table scan (which converts the partition data into table data). This operator will be different for different inputs. Option 2. seems like a better option.
          Hide
          Namit Jain added a comment -

          The possible options are to not allow the schema to be changed with LazyColumnarSerDe (only allow additions),
          or use partition metadata for inspectors.

          Show
          Namit Jain added a comment - The possible options are to not allow the schema to be changed with LazyColumnarSerDe (only allow additions), or use partition metadata for inspectors.
          Hide
          Namit Jain added a comment -

          Consider the following test:

          set hive.input.format = org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;

          create table partition_test_partitioned(key string, value string) partitioned by (dt string) stored as rcfile;

          alter table partition_test_partitioned set serde 'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe';
          insert overwrite table partition_test_partitioned partition(dt='1') select * from src where key = 238;

          alter table partition_test_partitioned change key key int;

          The query:
          select * from partition_test_partitioned where dt is not null;

          returns:

          50 val_238 1
          50 val_238 1

          This is due to the fact that the key column was serialized as a string column, and is now being read as a integer.

          Show
          Namit Jain added a comment - Consider the following test: set hive.input.format = org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; create table partition_test_partitioned(key string, value string) partitioned by (dt string) stored as rcfile; alter table partition_test_partitioned set serde 'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe'; insert overwrite table partition_test_partitioned partition(dt='1') select * from src where key = 238; alter table partition_test_partitioned change key key int; The query: select * from partition_test_partitioned where dt is not null; returns: 50 val_238 1 50 val_238 1 This is due to the fact that the key column was serialized as a string column, and is now being read as a integer.
          Namit Jain made changes -
          Field Original Value New Value
          Link This issue is related to HIVE-3824 [ HIVE-3824 ]
          Namit Jain created issue -

            People

            • Assignee:
              Namit Jain
              Reporter:
              Namit Jain
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development