Hive
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7794

Enable tests on Spark branch (4) [Sparch Branch]

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: spark-branch
    • Component/s: None
    • Labels:
      None

      Description

      This jira is to enable most of the tests below. If tests don't pass because of some unsupported feature, ensure that a JIRA exists and move on.

        vector_cast_constant.q,\
        vector_data_types.q,\
        vector_decimal_aggregate.q,\
        vector_left_outer_join.q,\
        vector_string_concat.q,\
        vectorization_12.q,\
        vectorization_13.q,\
        vectorization_14.q,\
        vectorization_15.q,\
        vectorization_9.q,\
        vectorization_part_project.q,\
        vectorization_short_regress.q,\
        vectorized_mapjoin.q,\
        vectorized_nested_mapjoin.q,\
        vectorized_ptf.q,\
        vectorized_shufflejoin.q,\
        vectorized_timestamp_funcs.q
      
      1. HIVE-7794-spark.patch
        1.50 MB
        Chinna Rao Lalam
      2. HIVE-7794.1-spark.patch
        545 kB
        Chinna Rao Lalam

        Activity

        Hide
        Xuefu Zhang added a comment -

        Patch committed to Spark branch. Thanks to Chinna for the contribution.

        Show
        Xuefu Zhang added a comment - Patch committed to Spark branch. Thanks to Chinna for the contribution.
        Hide
        Xuefu Zhang added a comment -

        +1

        Show
        Xuefu Zhang added a comment - +1
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12674145/HIVE-7794.1-spark.patch

        ERROR: -1 due to 2 failed/errored test(s), 6750 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
        org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parallel
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/202/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/202/console
        Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-202/

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 2 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12674145

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12674145/HIVE-7794.1-spark.patch ERROR: -1 due to 2 failed/errored test(s), 6750 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parallel Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/202/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/202/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-202/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12674145
        Hide
        Chinna Rao Lalam added a comment -

        Updated the patch.

        Patch doesnot contain below q files

        vector_string_concat.q
        vectorization_short_regress.q

        These queries executed as normal queries, will handle this in HIVE-7930

        vectorized_mapjoin.q
        vectorized_nested_mapjoin.q

        These queries will enable after enabling the map join, will handle this in HIVE-8425

        Show
        Chinna Rao Lalam added a comment - Updated the patch. Patch doesnot contain below q files vector_string_concat.q vectorization_short_regress.q These queries executed as normal queries, will handle this in HIVE-7930 vectorized_mapjoin.q vectorized_nested_mapjoin.q These queries will enable after enabling the map join, will handle this in HIVE-8425
        Hide
        Brock Noland added a comment -

        Patch looks good Chinna! Have you been able to tell if vectorization is actually enabled? e.g.: https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/tez/vector_cast_constant.q.out#L146

        Show
        Brock Noland added a comment - Patch looks good Chinna! Have you been able to tell if vectorization is actually enabled? e.g.: https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/tez/vector_cast_constant.q.out#L146
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12665766/HIVE-7794-spark.patch

        ERROR: -1 due to 3 failed/errored test(s), 6280 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2
        org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/108/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/108/console
        Test logs: http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-108/

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 3 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12665766

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12665766/HIVE-7794-spark.patch ERROR: -1 due to 3 failed/errored test(s), 6280 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2 org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/108/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/108/console Test logs: http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-108/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12665766
        Hide
        Chinna Rao Lalam added a comment -

        vectorization_short_regress.q,
        vectorized_timestamp_funcs.q,
        vectorized_nested_mapjoin.q will handle in HIVE-7930

        Show
        Chinna Rao Lalam added a comment - vectorization_short_regress.q, vectorized_timestamp_funcs.q, vectorized_nested_mapjoin.q will handle in HIVE-7930
        Hide
        Chinna Rao Lalam added a comment -

        vectorization_short_regress.q, vectorized_timestamp_funcs.q, vectorized_nested_mapjoin.q are failing.

        These 3 test case will handle in separate jira.

        Initial thought is because of UDF's and nested map join those are failing.

        Show
        Chinna Rao Lalam added a comment - vectorization_short_regress.q, vectorized_timestamp_funcs.q, vectorized_nested_mapjoin.q are failing. These 3 test case will handle in separate jira. Initial thought is because of UDF's and nested map join those are failing.

          People

          • Assignee:
            Chinna Rao Lalam
            Reporter:
            Brock Noland
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development