Hive
  1. Hive
  2. HIVE-4927

When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      set hive.auto.convert.join=true;
      set hive.auto.convert.join.noconditionaltask=true;
      EXPLAIN
      SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key) JOIN src1 y2 ON (x1.value = y2.value) GROUP BY x1.key;
      

      We will get a NPE from MetadataOnlyOptimizer. The reason is that the operator tree of the MapRedTask evaluating two MapJoins is

      TS1->MapJoin1->TS2->MapJoin2->...
      

      We should remove the TS2...

      1. HIVE-4927.D11811.1.patch
        17 kB
        Phabricator
      2. HIVE-4927.D11811.2.patch
        186 kB
        Phabricator
      3. HIVE-4927.D11811.3.patch
        74 kB
        Yin Huai
      4. HIVE-4927.D11811.3.patch
        74 kB
        Phabricator

        Issue Links

          Activity

          Hide
          Yin Huai added a comment -
          Show
          Yin Huai added a comment - https://reviews.facebook.net/D11811
          Hide
          Phabricator added a comment -

          yhuai requested code review of "HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed".

          Reviewers: JIRA

          remove the unnecessary TableScanOperator

          set hive.auto.convert.join=true;
          set hive.auto.convert.join.noconditionaltask=true;
          EXPLAIN
          SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key) JOIN src1 y2 ON (x1.value = y2.value) GROUP BY x1.key;

          We will get a NPE from MetadataOnlyOptimizer. The reason is that the operator tree of the MapRedTask evaluating two MapJoins is

          TS1->MapJoin1->TS2->MapJoin2->...

          We should remove the TS2...

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D11811

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          ql/src/test/results/clientpositive/multiMapJoin1.q.out

          MANAGE HERALD RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/28053/

          To: JIRA, yhuai

          Show
          Phabricator added a comment - yhuai requested code review of " HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed". Reviewers: JIRA remove the unnecessary TableScanOperator set hive.auto.convert.join=true; set hive.auto.convert.join.noconditionaltask=true; EXPLAIN SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key) JOIN src1 y2 ON (x1.value = y2.value) GROUP BY x1.key; We will get a NPE from MetadataOnlyOptimizer. The reason is that the operator tree of the MapRedTask evaluating two MapJoins is TS1->MapJoin1->TS2->MapJoin2->... We should remove the TS2... TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D11811 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java ql/src/test/results/clientpositive/multiMapJoin1.q.out MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/28053/ To: JIRA, yhuai
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12594067/HIVE-4927.D11811.1.patch

          ERROR: -1 due to 12 failed/errored test(s), 2651 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_join_union
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join28
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_star
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join33
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/179/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/179/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 12 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594067/HIVE-4927.D11811.1.patch ERROR: -1 due to 12 failed/errored test(s), 2651 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_join_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join28 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_star org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join33 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/179/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/179/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 12 tests failed This message is automatically generated.
          Hide
          Phabricator added a comment -

          yhuai updated the revision "HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed".

          • update results

          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D11811

          CHANGE SINCE LAST DIFF
          https://reviews.facebook.net/D11811?vs=36141&id=36183#toc

          AFFECTED FILES
          data/files/v1.txt
          data/files/v2.txt
          hcatalog/build-support/ant/checkstyle.xml
          metastore/src/model/package.jdo
          ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java
          ql/src/test/results/clientpositive/auto_join13.q.out
          ql/src/test/results/clientpositive/auto_join2.q.out
          ql/src/test/results/clientpositive/auto_join22.q.out
          ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          ql/src/test/results/clientpositive/join28.q.out
          ql/src/test/results/clientpositive/join32.q.out
          ql/src/test/results/clientpositive/join33.q.out
          ql/src/test/results/clientpositive/join_star.q.out
          ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          ql/src/test/results/clientpositive/multiMapJoin1.q.out
          ql/src/test/results/clientpositive/multi_join_union.q.out
          testutils/ptest2/README.md
          testutils/ptest2/src/main/java/org/apache/hive/ptest/api/client/PTestClient.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/api/request/TestStartRequest.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/api/server/TestExecutor.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/ExecutionPhase.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/HostExecutor.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/JIRAService.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/JUnitReportParser.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/PTest.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/ReportingPhase.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/QFileTestBatch.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/TestConfiguration.java
          testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/UnitTestBatch.java
          testutils/ptest2/src/main/resources/batch-exec.vm
          testutils/ptest2/src/main/resources/source-prep.vm
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.java
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.java
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportParser.java
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportingPhase.testExecute.approved.txt
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.java
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testBatch.approved.txt
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepGit.approved.txt
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepNone.approved.txt
          testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepSvn.approved.txt
          testutils/ptest2/src/test/resources/SomeTest-failure.xml
          testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml
          testutils/ptest2/src/test/resources/SomeTest-success.xml
          testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml
          testutils/ptest2/src/test/resources/test-outputs/SomeTest-truncated.xml
          testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/index_auth.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-hive.log
          testutils/ptest2/src/test/resources/test-outputs/index_auth.q-hive.log
          testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TestCliDriver.txt
          testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-hive.log
          testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TestCliDriver.txt
          testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-hive.log
          testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TestCliDriver.txt
          testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TestCliDriver.txt
          testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-hive.log

          To: JIRA, yhuai

          Show
          Phabricator added a comment - yhuai updated the revision " HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed". update results Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D11811 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D11811?vs=36141&id=36183#toc AFFECTED FILES data/files/v1.txt data/files/v2.txt hcatalog/build-support/ant/checkstyle.xml metastore/src/model/package.jdo ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join2.q.out ql/src/test/results/clientpositive/auto_join22.q.out ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out ql/src/test/results/clientpositive/join28.q.out ql/src/test/results/clientpositive/join32.q.out ql/src/test/results/clientpositive/join33.q.out ql/src/test/results/clientpositive/join_star.q.out ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out ql/src/test/results/clientpositive/mapjoin_subquery.q.out ql/src/test/results/clientpositive/mapjoin_subquery2.q.out ql/src/test/results/clientpositive/multiMapJoin1.q.out ql/src/test/results/clientpositive/multi_join_union.q.out testutils/ptest2/README.md testutils/ptest2/src/main/java/org/apache/hive/ptest/api/client/PTestClient.java testutils/ptest2/src/main/java/org/apache/hive/ptest/api/request/TestStartRequest.java testutils/ptest2/src/main/java/org/apache/hive/ptest/api/server/TestExecutor.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/ExecutionPhase.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/HostExecutor.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/JIRAService.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/JUnitReportParser.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/PTest.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/ReportingPhase.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/QFileTestBatch.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/TestConfiguration.java testutils/ptest2/src/main/java/org/apache/hive/ptest/execution/conf/UnitTestBatch.java testutils/ptest2/src/main/resources/batch-exec.vm testutils/ptest2/src/main/resources/source-prep.vm testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.java testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.java testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportParser.java testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportingPhase.testExecute.approved.txt testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.java testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testBatch.approved.txt testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepGit.approved.txt testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepNone.approved.txt testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepSvn.approved.txt testutils/ptest2/src/test/resources/SomeTest-failure.xml testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml testutils/ptest2/src/test/resources/SomeTest-success.xml testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml testutils/ptest2/src/test/resources/test-outputs/SomeTest-truncated.xml testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/index_auth.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-hive.log testutils/ptest2/src/test/resources/test-outputs/index_auth.q-hive.log testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TestCliDriver.txt testutils/ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-hive.log testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TestCliDriver.txt testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-hive.log testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TestCliDriver.txt testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TestCliDriver.txt testutils/ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-hive.log To: JIRA, yhuai
          Hide
          Phabricator added a comment -

          yhuai updated the revision "HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed".

          generate the correct diff. The last one is not based on the latest trunk

          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D11811

          CHANGE SINCE LAST DIFF
          https://reviews.facebook.net/D11811?vs=36183&id=36189#toc

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          ql/src/test/results/clientpositive/auto_join13.q.out
          ql/src/test/results/clientpositive/auto_join2.q.out
          ql/src/test/results/clientpositive/auto_join22.q.out
          ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          ql/src/test/results/clientpositive/join28.q.out
          ql/src/test/results/clientpositive/join32.q.out
          ql/src/test/results/clientpositive/join33.q.out
          ql/src/test/results/clientpositive/join_star.q.out
          ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          ql/src/test/results/clientpositive/multiMapJoin1.q.out
          ql/src/test/results/clientpositive/multi_join_union.q.out

          To: JIRA, yhuai

          Show
          Phabricator added a comment - yhuai updated the revision " HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed". generate the correct diff. The last one is not based on the latest trunk Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D11811 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D11811?vs=36183&id=36189#toc AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join2.q.out ql/src/test/results/clientpositive/auto_join22.q.out ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out ql/src/test/results/clientpositive/join28.q.out ql/src/test/results/clientpositive/join32.q.out ql/src/test/results/clientpositive/join33.q.out ql/src/test/results/clientpositive/join_star.q.out ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out ql/src/test/results/clientpositive/mapjoin_subquery.q.out ql/src/test/results/clientpositive/mapjoin_subquery2.q.out ql/src/test/results/clientpositive/multiMapJoin1.q.out ql/src/test/results/clientpositive/multi_join_union.q.out To: JIRA, yhuai
          Hide
          Yin Huai added a comment -

          HIVE-4927.D11811.2.patch is not based on the latest trunk. Please ignore it...

          Show
          Yin Huai added a comment - HIVE-4927 .D11811.2.patch is not based on the latest trunk. Please ignore it...
          Hide
          Yin Huai added a comment -

          Update test results. The hidden TableScanOperator will introduce the indent in the result of EXPLAIN... Since those TS does not have a conf, there is no other info related to the hidden TS..

          Show
          Yin Huai added a comment - Update test results. The hidden TableScanOperator will introduce the indent in the result of EXPLAIN... Since those TS does not have a conf, there is no other info related to the hidden TS..
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12594200/HIVE-4927.D11811.3.patch

          ERROR: -1 due to 2 failed/errored test(s), 2651 tests executed
          Failed tests:

          org.apache.hcatalog.pig.TestHCatLoader.testProjectionsBasic
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/187/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/187/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594200/HIVE-4927.D11811.3.patch ERROR: -1 due to 2 failed/errored test(s), 2651 tests executed Failed tests: org.apache.hcatalog.pig.TestHCatLoader.testProjectionsBasic org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/187/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/187/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Yin Huai added a comment -

          These two failed tests pass in my laptop.

          Show
          Yin Huai added a comment - These two failed tests pass in my laptop.
          Hide
          Yin Huai added a comment -

          Let's trigger the precommit tests again since the queue is short.

          Show
          Yin Huai added a comment - Let's trigger the precommit tests again since the queue is short.
          Hide
          Phabricator added a comment -

          ashutoshc has accepted the revision "HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed".

          +1

          REVISION DETAIL
          https://reviews.facebook.net/D11811

          BRANCH
          HIVE-4927

          ARCANIST PROJECT
          hive

          To: JIRA, ashutoshc, yhuai

          Show
          Phabricator added a comment - ashutoshc has accepted the revision " HIVE-4927 [jira] When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed". +1 REVISION DETAIL https://reviews.facebook.net/D11811 BRANCH HIVE-4927 ARCANIST PROJECT hive To: JIRA, ashutoshc, yhuai
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12594308/HIVE-4927.D11811.3.patch

          SUCCESS: +1 2652 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/189/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/189/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594308/HIVE-4927.D11811.3.patch SUCCESS: +1 2652 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/189/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/189/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Yin!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Yin!
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

            People

            • Assignee:
              Yin Huai
              Reporter:
              Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development