Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17148

Incorrect result for Hive join query with COALESCE in WHERE condition

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 3.0.0
    • Component/s: CBO
    • Labels:
      None

      Description

      The issue exists in Hive-2.1. In Hive-1.2 the query works fine with cbo enabled:

      STEPS TO REPRODUCE:

      Step 1: Create a table ct1
      create table ct1 (a1 string,b1 string);
      
      Step 2: Create a table ct2
      create table ct2 (a2 string);
      
      Step 3 : Insert following data into table ct1
      insert into table ct1 (a1) values ('1');
      
      Step 4 : Insert following data into table ct2
      insert into table ct2 (a2) values ('1');
      
      Step 5 : Execute the following query 
      select * from ct1 c1, ct2 c2 where COALESCE(a1,b1)=a2;
      

      ACTUAL RESULT:

      The query returns nothing;
      

      EXPECTED RESULT:

      1       NULL    1
      

      The issue seems to be because of the incorrect query plan. In the plan we can see:
      predicate:(a1 is not null and b1 is not null)
      which does not look correct. As a result, it is filtering out all the rows is any column mentioned in the COALESCE has null value.
      Please find the query plan below:

      Plan optimized by CBO.
      
      Vertex dependency in root stage
      Map 1 <- Map 2 (BROADCAST_EDGE)
      
      Stage-0
        Fetch Operator
          limit:-1
          Stage-1
            Map 1
            File Output Operator [FS_10]
              Map Join Operator [MAPJOIN_15] (rows=1 width=4)
                Conds:SEL_2.COALESCE(_col0,_col1)=RS_7._col0(Inner),HybridGraceHashJoin:true,Output:["_col0","_col1","_col2"]
              <-Map 2 [BROADCAST_EDGE]
                BROADCAST [RS_7]
                  PartitionCols:_col0
                  Select Operator [SEL_5] (rows=1 width=1)
                    Output:["_col0"]
                    Filter Operator [FIL_14] (rows=1 width=1)
                      predicate:a2 is not null
                      TableScan [TS_3] (rows=1 width=1)
                        default@ct2,c2,Tbl:COMPLETE,Col:NONE,Output:["a2"]
              <-Select Operator [SEL_2] (rows=1 width=4)
                  Output:["_col0","_col1"]
                  Filter Operator [FIL_13] (rows=1 width=4)
                    predicate:(a1 is not null and b1 is not null)
                    TableScan [TS_0] (rows=1 width=4)
                      default@ct1,c1,Tbl:COMPLETE,Col:NONE,Output:["a1","b1"]
      

      This happens only if join is inner type, otherwise HiveJoinAddNotRule which creates this problem is skipped.

      1. HIVE-17148.1.patch
        22 kB
        Vlad Gudikov
      2. HIVE-17148.2.patch
        41 kB
        Vlad Gudikov
      3. HIVE-17148.3.patch
        84 kB
        Vlad Gudikov
      4. HIVE-17148.patch
        4 kB
        Vlad Gudikov

        Activity

        Hide
        allgoodok Vlad Gudikov added a comment -

        The thing is that while optimizing query HiveJoinAddNotRule which is checking if values that are part of filter are not null. The thing is that coalesce is working with null values, but tuples with null values are omitted.

        Show
        allgoodok Vlad Gudikov added a comment - The thing is that while optimizing query HiveJoinAddNotRule which is checking if values that are part of filter are not null. The thing is that coalesce is working with null values, but tuples with null values are omitted.
        Hide
        hiveqa Hive QA added a comment -

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12879177/HIVE-17148.patch

        ERROR: -1 due to no test(s) being added or modified.

        ERROR: -1 due to 15 failed/errored test(s), 11012 tests executed
        Failed tests:

        TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning] (batchId=32)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=15)
        org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=144)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=159)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage2] (batchId=156)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=157)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=168)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168)
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
        org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=128)
        org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge (batchId=206)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179)
        org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6153/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6153/console
        Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6153/

        Messages:

        Executing org.apache.hive.ptest.execution.TestCheckPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 15 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12879177 - PreCommit-HIVE-Build

        Show
        hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12879177/HIVE-17148.patch ERROR: -1 due to no test(s) being added or modified. ERROR: -1 due to 15 failed/errored test(s), 11012 tests executed Failed tests: TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning] (batchId=32) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=15) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=144) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage2] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=157) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=128) org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge (batchId=206) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6153/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6153/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6153/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed This message is automatically generated. ATTACHMENT ID: 12879177 - PreCommit-HIVE-Build
        Hide
        allgoodok Vlad Gudikov added a comment -

        Related test failures are due to changes in the plan as we do not create not null conjunctions for fields that are in filter but for expressions in filter as well.

        Show
        allgoodok Vlad Gudikov added a comment - Related test failures are due to changes in the plan as we do not create not null conjunctions for fields that are in filter but for expressions in filter as well.
        Hide
        allgoodok Vlad Gudikov added a comment - - edited

        ROOT-CAUSE:
        The problem was with the predicates that were created according to HiveJoinAddNotNullRule. This rule is creating predicates from fields that take part in join filter, no matter if this fields are used as parameters of functions or not.

        SOLUTION:
        Create predicate based on functions that take part in filters as well as fields. The point is to check if left part and right part of the filter is not null, not just fields that are part of the join filter. I.e we have to tables test1(a1 int, a2 int) and test2(b1). When we execute following query select * from ct1 c1 inner join ct2 c2 on (COALESCE(a1,b1)=a2); we get to predicates for filter operator:
        b1 is not null — right part
        a1 is not null and a2 is not null – left part

        Applying predicate for left part of join will result in data loss as we exclude rows with null fields. COALESCE is a good example for this case as the main purpose of COALESCE function is to get not null values from tables. To fix the data loss we need to check that coalesce won't bring us null values as we can't join nulls. My fix will check that left part and right part will look like:

        b1 is not null – right part (still checking fields on null condition)
        COALESCE(a1,a2) is not null (checking that whole function won't bring us null values)

        In next patch I'm going to change related failed tests with the fixed stage plans.

        Show
        allgoodok Vlad Gudikov added a comment - - edited ROOT-CAUSE: The problem was with the predicates that were created according to HiveJoinAddNotNullRule. This rule is creating predicates from fields that take part in join filter, no matter if this fields are used as parameters of functions or not. SOLUTION: Create predicate based on functions that take part in filters as well as fields. The point is to check if left part and right part of the filter is not null, not just fields that are part of the join filter. I.e we have to tables test1(a1 int, a2 int) and test2(b1) . When we execute following query select * from ct1 c1 inner join ct2 c2 on (COALESCE(a1,b1)=a2); we get to predicates for filter operator: b1 is not null — right part a1 is not null and a2 is not null – left part Applying predicate for left part of join will result in data loss as we exclude rows with null fields. COALESCE is a good example for this case as the main purpose of COALESCE function is to get not null values from tables. To fix the data loss we need to check that coalesce won't bring us null values as we can't join nulls. My fix will check that left part and right part will look like: b1 is not null – right part (still checking fields on null condition) COALESCE(a1,a2) is not null (checking that whole function won't bring us null values) In next patch I'm going to change related failed tests with the fixed stage plans.
        Hide
        ashutoshc Ashutosh Chauhan added a comment -

        Vlad Gudikov Can you add a testcase with your patch?

        Show
        ashutoshc Ashutosh Chauhan added a comment - Vlad Gudikov Can you add a testcase with your patch?
        Hide
        allgoodok Vlad Gudikov added a comment -

        Added patch with testcase

        Show
        allgoodok Vlad Gudikov added a comment - Added patch with testcase
        Hide
        hiveqa Hive QA added a comment -

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12880173/HIVE-17148.1.patch

        SUCCESS: +1 due to 1 test(s) being added or modified.

        ERROR: -1 due to 19 failed/errored test(s), 11145 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=23)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] (batchId=56)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning] (batchId=32)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=82)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=15)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=158)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=159)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage2] (batchId=156)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=157)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168)
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=236)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=236)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query24] (batchId=236)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query8] (batchId=236)
        org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=128)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179)
        org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6259/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6259/console
        Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6259/

        Messages:

        Executing org.apache.hive.ptest.execution.TestCheckPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 19 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12880173 - PreCommit-HIVE-Build

        Show
        hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880173/HIVE-17148.1.patch SUCCESS: +1 due to 1 test(s) being added or modified. ERROR: -1 due to 19 failed/errored test(s), 11145 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=23) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] (batchId=56) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning] (batchId=32) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=82) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=15) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage2] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=157) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=236) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=236) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query24] (batchId=236) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query8] (batchId=236) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=128) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6259/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6259/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6259/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 19 tests failed This message is automatically generated. ATTACHMENT ID: 12880173 - PreCommit-HIVE-Build
        Hide
        ashutoshc Ashutosh Chauhan added a comment -

        +1

        Show
        ashutoshc Ashutosh Chauhan added a comment - +1
        Hide
        allgoodok Vlad Gudikov added a comment -

        Ashutosh Chauhan today I will upload another patch with fixed related tests

        Show
        allgoodok Vlad Gudikov added a comment - Ashutosh Chauhan today I will upload another patch with fixed related tests
        Hide
        allgoodok Vlad Gudikov added a comment -

        Uploaded new patch with fixed tests

        Show
        allgoodok Vlad Gudikov added a comment - Uploaded new patch with fixed tests
        Hide
        hiveqa Hive QA added a comment -

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12880974/HIVE-17148.2.patch

        SUCCESS: +1 due to 1 test(s) being added or modified.

        ERROR: -1 due to 32 failed/errored test(s), 11000 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=243)
        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=243)
        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=243)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join] (batchId=51)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlated_join_keys] (batchId=26)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_mapjoin] (batchId=37)
        org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=143)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=159)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_15] (batchId=160)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=154)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_mapjoin] (batchId=153)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=169)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only] (batchId=170)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169)
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100)
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query30] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query49] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query50] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query5] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81] (batchId=235)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85] (batchId=235)
        org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[annotate_stats_join] (batchId=123)
        org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_15] (batchId=133)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180)
        org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180)
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6319/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6319/console
        Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6319/

        Messages:

        Executing org.apache.hive.ptest.execution.TestCheckPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 32 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12880974 - PreCommit-HIVE-Build

        Show
        hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880974/HIVE-17148.2.patch SUCCESS: +1 due to 1 test(s) being added or modified. ERROR: -1 due to 32 failed/errored test(s), 11000 tests executed Failed tests: org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=243) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join] (batchId=51) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlated_join_keys] (batchId=26) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_mapjoin] (batchId=37) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=143) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_15] (batchId=160) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_mapjoin] (batchId=153) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only] (batchId=170) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query30] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query49] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query50] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query5] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85] (batchId=235) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[annotate_stats_join] (batchId=123) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_15] (batchId=133) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6319/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6319/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6319/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 32 tests failed This message is automatically generated. ATTACHMENT ID: 12880974 - PreCommit-HIVE-Build
        Hide
        hiveqa Hive QA added a comment -

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12881456/HIVE-17148.3.patch

        SUCCESS: +1 due to 1 test(s) being added or modified.

        ERROR: -1 due to 14 failed/errored test(s), 11003 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1] (batchId=240)
        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=243)
        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=243)
        org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=243)
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal] (batchId=9)
        org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=159)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=169)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only] (batchId=170)
        org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169)
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
        org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=235)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180)
        org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180)
        org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180)
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6357/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6357/console
        Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6357/

        Messages:

        Executing org.apache.hive.ptest.execution.TestCheckPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 14 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12881456 - PreCommit-HIVE-Build

        Show
        hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12881456/HIVE-17148.3.patch SUCCESS: +1 due to 1 test(s) being added or modified. ERROR: -1 due to 14 failed/errored test(s), 11003 tests executed Failed tests: org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1] (batchId=240) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=243) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal] (batchId=9) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=159) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only] (batchId=170) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=235) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6357/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6357/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6357/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 14 tests failed This message is automatically generated. ATTACHMENT ID: 12881456 - PreCommit-HIVE-Build
        Hide
        ashutoshc Ashutosh Chauhan added a comment -

        Pushed to master. Thanks, Vlad!

        Show
        ashutoshc Ashutosh Chauhan added a comment - Pushed to master. Thanks, Vlad!

          People

          • Assignee:
            allgoodok Vlad Gudikov
            Reporter:
            allgoodok Vlad Gudikov
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development