Hive
  1. Hive
  2. HIVE-5771

Constant propagation optimizer for Hive

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Currently there is no constant folding/propagation optimizer, all expressions are evaluated at runtime.

      HIVE-2470 did a great job on evaluating constants on UDF initializing phase, however, it is still a runtime evaluation and it doesn't propagate constants from a subquery to outside.

      It may reduce I/O and accelerate process if we introduce such an optimizer.

      1. HIVE-5771.patch
        323 kB
        Ted Xu
      2. HIVE-5771.8.patch
        677 kB
        Ashutosh Chauhan
      3. HIVE-5771.7.patch
        567 kB
        Ashutosh Chauhan
      4. HIVE-5771.6.patch
        2.43 MB
        Ted Xu
      5. HIVE-5771.5.patch
        2.42 MB
        Ted Xu
      6. HIVE-5771.4.patch
        2.64 MB
        Ted Xu
      7. HIVE-5771.3.patch
        2.64 MB
        Ted Xu
      8. HIVE-5771.2.patch
        2.98 MB
        Ted Xu
      9. HIVE-5771.1.patch
        3.00 MB
        Ted Xu

        Activity

        Hide
        Xuefu Zhang added a comment -

        Ted Xu Thank you for your patch. Please rename the patch according to the convention so that test can run, something like HIVE-5771.patch, HIVE-5771.1.patch, and so on.

        Show
        Xuefu Zhang added a comment - Ted Xu Thank you for your patch. Please rename the patch according to the convention so that test can run, something like HIVE-5771 .patch, HIVE-5771 .1.patch, and so on.
        Hide
        Eric Hanson added a comment -

        Does this do constant folding before the vectorization process? If it can do that, then more queries will vectorize.

        Please put up a ReviewBoard entry for your patch to make it easier to review.

        Thanks,
        Eric

        Show
        Eric Hanson added a comment - Does this do constant folding before the vectorization process? If it can do that, then more queries will vectorize. Please put up a ReviewBoard entry for your patch to make it easier to review. Thanks, Eric
        Hide
        Ted Xu added a comment -

        Thanks Xuefu! I already renamed it.

        Show
        Ted Xu added a comment - Thanks Xuefu! I already renamed it.
        Hide
        Ted Xu added a comment -

        Eric, thank you for your comment.

        Yes, the constant propagate is an logical plan optimizer so it will process before vectorization process. But I'm not sure how constant folding can benefit vectorized execution. If there is anything I can do to improve it please let me know, thanks!

        I created a review request here: https://reviews.apache.org/r/15561/

        Show
        Ted Xu added a comment - Eric, thank you for your comment. Yes, the constant propagate is an logical plan optimizer so it will process before vectorization process. But I'm not sure how constant folding can benefit vectorized execution. If there is anything I can do to improve it please let me know, thanks! I created a review request here: https://reviews.apache.org/r/15561/
        Hide
        Eric Hanson added a comment -

        Right now, a query like this:

        select ...
        from T
        where a = 2 * 2;

        will not vectorize. This is because the "=(column,scalar)" operator looks in the syntax tree to find variable expression result or column on the left and a single scalar on the right. If the constant folder can turn 2 * 2 into the integer literal 4, then the vectorization code can process the "a = 4" filter.

        Being able to fold CAST(constant-expression AS type) will also help, because people use this, especially for casting a string literal to a timestamp.

        Show
        Eric Hanson added a comment - Right now, a query like this: select ... from T where a = 2 * 2; will not vectorize. This is because the "=(column,scalar)" operator looks in the syntax tree to find variable expression result or column on the left and a single scalar on the right. If the constant folder can turn 2 * 2 into the integer literal 4, then the vectorization code can process the "a = 4" filter. Being able to fold CAST(constant-expression AS type) will also help, because people use this, especially for casting a string literal to a timestamp.
        Hide
        Eric Hanson added a comment -

        Ted,

        This looks excellent. Please address my comments on ReviewBoard. They are mostly suggestions about code comments and style. This patch will definitely help more queries vectorize and reduce some work needed for the translation from a standard expression to a VectorExpression.

        One thing you may want to think about is that right now there is no such thing as a Timestamp literal, I don't think. But you will create Timestamp constants in your folding. I don't think that will cause any problems, but it might. It is something to think about. E.g. do you need an test that casts a string to a timestamp?

        Eric

        Show
        Eric Hanson added a comment - Ted, This looks excellent. Please address my comments on ReviewBoard. They are mostly suggestions about code comments and style. This patch will definitely help more queries vectorize and reduce some work needed for the translation from a standard expression to a VectorExpression. One thing you may want to think about is that right now there is no such thing as a Timestamp literal, I don't think. But you will create Timestamp constants in your folding. I don't think that will cause any problems, but it might. It is something to think about. E.g. do you need an test that casts a string to a timestamp? Eric
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12613993/HIVE-5771.patch

        ERROR: -1 due to 92 failed/errored test(s), 4616 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_cast
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_comparison
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_udf1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_column_access_stats
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog_dp
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_logical
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_ppd
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_empty
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_file_format
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_multiple
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_unused
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_update
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_auto_partitioned
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale_partitioned
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_const_type
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input25
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input26
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input42
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part0
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join38
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_view
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_ppd
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_move_tasks_share_dependencies
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonmr_fetch
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_predicate_pushdown
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_outer_join5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_repeated_alias
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_udf_case
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_udf_col
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_union_view
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_set_processor_namespaces
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_25
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subq_where_serialization
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_in_file
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union33
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view
        org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_add
        org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_format
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_cast1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_input6
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join2
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join3
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join4
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join5
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_join8
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf4
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf6
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_case
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_when
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/321/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/321/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 92 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12613993

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12613993/HIVE-5771.patch ERROR: -1 due to 92 failed/errored test(s), 4616 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_cast org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_comparison org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_udf1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_column_access_stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog_dp org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_logical org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_ppd org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_empty org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_file_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_multiple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_unused org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_update org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_auto_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_const_type org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input42 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_part9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join38 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_ppd org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_move_tasks_share_dependencies org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonmr_fetch org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_predicate_pushdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_outer_join5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_repeated_alias org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_udf_case org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_udf_col org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_union_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_set_processor_namespaces org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subq_where_serialization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_in_file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union33 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_add org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver_udf_example_format org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges org.apache.hadoop.hive.ql.parse.TestParse.testParse_cast1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6 org.apache.hadoop.hive.ql.parse.TestParse.testParse_input6 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_join8 org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf6 org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_case org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_when Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/321/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/321/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 92 tests failed This message is automatically generated. ATTACHMENT ID: 12613993
        Hide
        Ted Xu added a comment -

        Hi Eric,

        Thank you for your reviewing. I already updated the diff file according to your suggestions.

        With regard to timestamp literal and cast constants, this optimizer will fold them like other primitive types. I tested timestamp and date, it look good to me. I'm sorry I haven't got chance to merge the tests into HIVE-5771.1.patch. I will add them later.

        Show
        Ted Xu added a comment - Hi Eric, Thank you for your reviewing. I already updated the diff file according to your suggestions. With regard to timestamp literal and cast constants, this optimizer will fold them like other primitive types. I tested timestamp and date, it look good to me. I'm sorry I haven't got chance to merge the tests into HIVE-5771 .1.patch. I will add them later.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614281/HIVE-5771.1.patch

        ERROR: -1 due to 17 failed/errored test(s), 4612 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like_view
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_alter
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_18
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_in_file
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unset_table_view_property
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/334/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/334/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 17 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12614281

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614281/HIVE-5771.1.patch ERROR: -1 due to 17 failed/errored test(s), 4612 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_alter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_in_file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unset_table_view_property org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/334/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/334/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 17 tests failed This message is automatically generated. ATTACHMENT ID: 12614281
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614382/HIVE-5771.2.patch

        ERROR: -1 due to 49 failed/errored test(s), 4613 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_predicate_pushdown
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_outer_join4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_allchildsarenull
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_18
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_19
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_20
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_between
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_find_in_set
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_reverse
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_0
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_16
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_not
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_math_funcs
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_string_funcs
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
        org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/342/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/342/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 49 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12614382

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614382/HIVE-5771.2.patch ERROR: -1 due to 49 failed/errored test(s), 4613 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_predicate_pushdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_outer_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_allchildsarenull org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_between org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_find_in_set org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_reverse org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_not org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_math_funcs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_string_funcs org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/342/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/342/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 49 tests failed This message is automatically generated. ATTACHMENT ID: 12614382
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614632/HIVE-5771.3.patch

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/367/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/367/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
        + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + cd /data/hive-ptest/working/
        + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-367/source-prep.txt
        + [[ false == \t\r\u\e ]]
        + mkdir -p maven ivy
        + [[ svn = \s\v\n ]]
        + [[ -n '' ]]
        + [[ -d apache-svn-trunk-source ]]
        + [[ ! -d apache-svn-trunk-source/.svn ]]
        + [[ ! -d apache-svn-trunk-source ]]
        + cd apache-svn-trunk-source
        + svn revert -R .
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java'
        ++ awk '{print $2}'
        ++ egrep -v '^X|^Performing status on external'
        ++ svn status --no-ignore
        + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/assembly/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
        + svn update
        
        Fetching external item into 'hcatalog/src/test/e2e/harness'
        External at revision 1543651.
        
        At revision 1543651.
        + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
        + patchFilePath=/data/hive-ptest/working/scratch/build.patch
        + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
        + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
        + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
        The patch does not appear to apply with p0, p1, or p2
        + exit 1
        '
        

        This message is automatically generated.

        ATTACHMENT ID: 12614632

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614632/HIVE-5771.3.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/367/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/367/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-367/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/assembly/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1543651. At revision 1543651. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12614632
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614749/HIVE-5771.4.patch

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/371/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/371/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
        + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + cd /data/hive-ptest/working/
        + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-371/source-prep.txt
        + [[ false == \t\r\u\e ]]
        + mkdir -p maven ivy
        + [[ svn = \s\v\n ]]
        + [[ -n '' ]]
        + [[ -d apache-svn-trunk-source ]]
        + [[ ! -d apache-svn-trunk-source/.svn ]]
        + [[ ! -d apache-svn-trunk-source ]]
        + cd apache-svn-trunk-source
        + svn revert -R .
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/InStream.java'
        ++ awk '{print $2}'
        ++ egrep -v '^X|^Performing status on external'
        ++ svn status --no-ignore
        + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/assembly/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen service/target contrib/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
        + svn update
        
        Fetching external item into 'hcatalog/src/test/e2e/harness'
        External at revision 1543817.
        
        At revision 1543814.
        + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
        + patchFilePath=/data/hive-ptest/working/scratch/build.patch
        + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
        + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
        + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
        The patch does not appear to apply with p0, p1, or p2
        + exit 1
        '
        

        This message is automatically generated.

        ATTACHMENT ID: 12614749

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614749/HIVE-5771.4.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/371/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/371/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-371/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/InStream.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/assembly/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen service/target contrib/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1543817. At revision 1543814. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12614749
        Hide
        Eric Hanson added a comment -

        Where does this patch stand? Ted, are you going to move it forward?

        Show
        Eric Hanson added a comment - Where does this patch stand? Ted, are you going to move it forward?
        Hide
        Ted Xu added a comment -

        Hi Eric,
        Thank you for your concern. I'm sorry the work have been delayed by some other stuff. I will move it forward ASAP.

        Show
        Ted Xu added a comment - Hi Eric, Thank you for your concern. I'm sorry the work have been delayed by some other stuff. I will move it forward ASAP.
        Hide
        Eric Hanson added a comment -

        Glad to hear it!

        Show
        Eric Hanson added a comment - Glad to hear it!
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12622188/HIVE-5771.5.patch

        ERROR: -1 due to 10 failed/errored test(s), 4908 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_udf
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notexists_having
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin_having
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_div0
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 10 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12622188

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12622188/HIVE-5771.5.patch ERROR: -1 due to 10 failed/errored test(s), 4908 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_udf org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notexists_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_div0 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed This message is automatically generated. ATTACHMENT ID: 12622188
        Hide
        Eric Hanson added a comment -

        For vectorization_div0, there is the following diff here: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vectorization_div0/

        < expr: ((cdouble >= -500) and (cdouble < -199))

        > expr: ((cdouble >= (- 500)) and (cdouble < (- 199)))

        The difference is that the unary-minus was constant-folded so one blank is missing. This is good and not a problem. The query still vectorizes. The vectorization code has a special-purpose constant folder in it now to do this. Your code will make that unnecessary. You can safely regenerate the .q.out file for this test.

        Show
        Eric Hanson added a comment - For vectorization_div0, there is the following diff here: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/843/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vectorization_div0/ < expr: ((cdouble >= -500) and (cdouble < -199)) — > expr: ((cdouble >= (- 500)) and (cdouble < (- 199))) The difference is that the unary-minus was constant-folded so one blank is missing. This is good and not a problem. The query still vectorizes. The vectorization code has a special-purpose constant folder in it now to do this. Your code will make that unnecessary. You can safely regenerate the .q.out file for this test.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12623285/HIVE-5771.6.patch

        ERROR: -1 due to 1 failed/errored test(s), 4931 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/930/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/930/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12623285

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12623285/HIVE-5771.6.patch ERROR: -1 due to 1 failed/errored test(s), 4931 tests executed Failed tests: org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/930/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/930/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12623285
        Hide
        Eric Hanson added a comment -

        Ted, can you please post the latest patch to Review Board?

        Show
        Eric Hanson added a comment - Ted, can you please post the latest patch to Review Board?
        Hide
        Ted Xu added a comment -

        Eric, I've updated the diff file on https://reviews.apache.org/r/15561/. Thanks.

        Show
        Ted Xu added a comment - Eric, I've updated the diff file on https://reviews.apache.org/r/15561/ . Thanks.
        Hide
        Ashutosh Chauhan added a comment - - edited

        Pretty good work, Ted. Hive is in need of this optimization for long time. Thanks for taking it up.
        I scanned the patch. Mostly looking at .q.out changes. Most of them look are correct, except following :

        • smb_mapjoin_18.q : Seems like a Map only job has turned into MR job.
        • smb_mapjoin_25.q : extra MR stage got introduced
        • groupby_sort_1.q --> extra MR stage got introduced
        • groupby_sort_skew_1.q --> extra MR stage got introduced
        • udf_between.q --> betweeen 2 and '3' got optimized away. Here types don't match, shouldn't this instead have optimized into always false filter?
        • decimal.q - optimization is turned off. Any particular reason?
        • pcr.q - optimization is turned off. Any particular reason?

        I haven't looked at code changes yet. Will be looking at those soon.

        Show
        Ashutosh Chauhan added a comment - - edited Pretty good work, Ted. Hive is in need of this optimization for long time. Thanks for taking it up. I scanned the patch. Mostly looking at .q.out changes. Most of them look are correct, except following : smb_mapjoin_18.q : Seems like a Map only job has turned into MR job. smb_mapjoin_25.q : extra MR stage got introduced groupby_sort_1.q --> extra MR stage got introduced groupby_sort_skew_1.q --> extra MR stage got introduced udf_between.q --> betweeen 2 and '3' got optimized away. Here types don't match, shouldn't this instead have optimized into always false filter? decimal.q - optimization is turned off. Any particular reason? pcr.q - optimization is turned off. Any particular reason? I haven't looked at code changes yet. Will be looking at those soon.
        Hide
        Ted Xu added a comment -

        Hi Ashutosh,

        Your points are valid, thanks! Here is my thinking of those issues:

        • smb_mapjoin_18.q & smb_mapjoin_25.q: those problems are introduced by constant propagate optimizer (CPO) conflicting with Bucketing Sorting ReduceSink Optimizer (BSRO). I tried apply BSRO before CPO and the issue seems fixed.
        • groupby_sort_1.q & groupby_sort_skew_1.q: those are because of CPO conflicting with Groupby Optimizer (GO), apply it before CPO also fixes issue. In fact I'm wondering if it is safe to reorder those optimizers, making it GO->BSRO->CPO.
        • decimal.q & pcr.q: I disabled these two cases because of some issue I still not figured out. My local machine told me to patch a piece of output data like '0.0040' to '0,004', but it is still '0.0040' in hudson server. I guess it is an environment issue.

        I will update the patch as soon as I validated the above fixes.

        Show
        Ted Xu added a comment - Hi Ashutosh , Your points are valid, thanks! Here is my thinking of those issues: smb_mapjoin_18.q & smb_mapjoin_25.q: those problems are introduced by constant propagate optimizer (CPO) conflicting with Bucketing Sorting ReduceSink Optimizer (BSRO) . I tried apply BSRO before CPO and the issue seems fixed. groupby_sort_1.q & groupby_sort_skew_1.q: those are because of CPO conflicting with Groupby Optimizer (GO) , apply it before CPO also fixes issue. In fact I'm wondering if it is safe to reorder those optimizers, making it GO->BSRO->CPO. decimal.q & pcr.q: I disabled these two cases because of some issue I still not figured out. My local machine told me to patch a piece of output data like '0.0040' to '0,004', but it is still '0.0040' in hudson server. I guess it is an environment issue. I will update the patch as soon as I validated the above fixes.
        Hide
        Ashutosh Chauhan added a comment -

        Ted Xu Are you still working on this? I would like to see this get in, since its pretty useful optimization.

        Show
        Ashutosh Chauhan added a comment - Ted Xu Are you still working on this? I would like to see this get in, since its pretty useful optimization.
        Hide
        Ashutosh Chauhan added a comment -

        Ted Xu I have rebased your patch on latest trunk. I ran few tests and it failed few of those like : smb_mapjoin9.q,smb_mapjoin_25.q,cluster.q,ppd_clusterby.q
        I havent ran all tests yet.
        Let me know if you will continue working on this issue ?

        Show
        Ashutosh Chauhan added a comment - Ted Xu I have rebased your patch on latest trunk. I ran few tests and it failed few of those like : smb_mapjoin9.q,smb_mapjoin_25.q,cluster.q,ppd_clusterby.q I havent ran all tests yet. Let me know if you will continue working on this issue ?
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12640915/HIVE-5771.7.patch

        ERROR: -1 due to 106 failed/errored test(s), 5410 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketizedhiveinputformat_auto
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_rearrange
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_unqual1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_unqual3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters_overlap
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_date
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_quotedid_skew
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt17
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt19
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt20
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_16
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_17
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_25
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_in
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_mapjoin
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_math_funcs
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_bucketmapjoin1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_map_operators
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_quotedid_smb
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_smb_mapjoin_8
        org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs
        

        Test results: http://bigtop01.cloudera.org:8080/job/precommit-hive/23/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/precommit-hive/23/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 106 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12640915

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12640915/HIVE-5771.7.patch ERROR: -1 due to 106 failed/errored test(s), 5410 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketizedhiveinputformat_auto org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_rearrange org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_unqual1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_unqual3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters_overlap org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_quotedid_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_bucketmapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_map_operators org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_quotedid_smb org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_smb_mapjoin_8 org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs Test results: http://bigtop01.cloudera.org:8080/job/precommit-hive/23/testReport Console output: http://bigtop01.cloudera.org:8080/job/precommit-hive/23/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 106 tests failed This message is automatically generated. ATTACHMENT ID: 12640915
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12640969/HIVE-5771.8.patch

        ERROR: -1 due to 10 failed/errored test(s), 5410 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_date
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
        org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
        org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/14/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/14/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 10 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12640969

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12640969/HIVE-5771.8.patch ERROR: -1 due to 10 failed/errored test(s), 5410 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cluster org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_clusterby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/14/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/14/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed This message is automatically generated. ATTACHMENT ID: 12640969

          People

          • Assignee:
            Ted Xu
            Reporter:
            Ted Xu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development