Hive
  1. Hive
  2. HIVE-6828

Hive tez bucket map join conversion interferes with map join conversion

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0, 0.14.0
    • Fix Version/s: 0.14.0, 0.13.1
    • Component/s: Tez
    • Labels:
      None

      Description

      The issue is that bucket count is used for checking the scaled down size of the hash tables but is used later on to convert to the map join as well which may be incorrect in cases where the entire hash table does not fit in the specified size.

      1. HIVE-6828.1.patch
        3 kB
        Vikram Dixit K
      2. HIVE-6828.2.patch
        3 kB
        Gunther Hagleitner

        Activity

        Hide
        Thejas M Nair added a comment -

        This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Thejas M Nair added a comment - This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.
        Hide
        Gunther Hagleitner added a comment -

        Committed to trunk. Thanks Vikram!

        Show
        Gunther Hagleitner added a comment - Committed to trunk. Thanks Vikram!
        Hide
        Gunther Hagleitner added a comment -

        No new test failures (build 100 had the same errors - plus one more).

        Show
        Gunther Hagleitner added a comment - No new test failures (build 100 had the same errors - plus one more).
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12642970/HIVE-6828.2.patch

        ERROR: -1 due to 6 failed/errored test(s), 5427 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 6 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12642970

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12642970/HIVE-6828.2.patch ERROR: -1 due to 6 failed/errored test(s), 5427 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed This message is automatically generated. ATTACHMENT ID: 12642970
        Hide
        Gunther Hagleitner added a comment -

        Uploading same patch to run test suite one more time.

        Show
        Gunther Hagleitner added a comment - Uploading same patch to run test suite one more time.
        Hide
        Gunther Hagleitner added a comment -

        +1

        Show
        Gunther Hagleitner added a comment - +1
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12638424/HIVE-6828.1.patch

        SUCCESS: +1 5518 tests passed

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2112/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2112/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        ATTACHMENT ID: 12638424

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12638424/HIVE-6828.1.patch SUCCESS: +1 5518 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2112/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2112/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12638424

          People

          • Assignee:
            Vikram Dixit K
            Reporter:
            Vikram Dixit K
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development