Hive
  1. Hive
  2. HIVE-6551

group by after join with skew join optimization references invalid task sometimes

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      For example,

      hive> set hive.auto.convert.join = true;
      hive> set hive.optimize.skewjoin = true;
      hive> set hive.skewjoin.key = 3;
      hive> 
          > EXPLAIN FROM 
          > (SELECT src.* FROM src) x
          > JOIN 
          > (SELECT src.* FROM src) Y
          > ON (x.key = Y.key)
          > SELECT sum(hash(Y.key)), sum(hash(Y.value));
      OK
      STAGE DEPENDENCIES:
        Stage-8 is a root stage
        Stage-6 depends on stages: Stage-8
        Stage-5 depends on stages: Stage-6 , consists of Stage-4, Stage-2
        Stage-4
        Stage-2 depends on stages: Stage-4, Stage-1
        Stage-0 is a root stage
      ...
      

      Stage-2 references not-existing Stage-1

        Activity

        Hide
        Harish Butani added a comment -

        ported to 0.13

        Show
        Harish Butani added a comment - ported to 0.13
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Navis!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Navis!
        Hide
        Ashutosh Chauhan added a comment -

        +1

        Show
        Ashutosh Chauhan added a comment - +1
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12632789/HIVE-6551.1.patch.txt

        ERROR: -1 due to 3 failed/errored test(s), 5358 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5
        org.apache.hive.beeline.TestSchemaTool.testSchemaInit
        org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1637/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1637/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 3 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12632789

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12632789/HIVE-6551.1.patch.txt ERROR: -1 due to 3 failed/errored test(s), 5358 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5 org.apache.hive.beeline.TestSchemaTool.testSchemaInit org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1637/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1637/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12632789

          People

          • Assignee:
            Navis
            Reporter:
            Navis
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development