Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.1, 2.1.0
    • Hive
    • None

    Description

      For a given job, one task may reuse other task's object cache (plan cache) such as MapJoinOperator. This is fine. But if we have some dirty states left over, it may cause issue like wrong results.

      Attachments

        1. HIVE-13126.1.patch
          1 kB
          Wei Zheng
        2. HIVE-13126.2.patch
          1 kB
          Wei Zheng
        3. HIVE-13126.3.patch
          1 kB
          Wei Zheng

        Activity

          wzheng Wei Zheng added a comment -

          Vikram Dixit K Can you take a look?

          wzheng Wei Zheng added a comment - Vikram Dixit K Can you take a look?
          vikram.dixit Vikram Dixit K added a comment -

          +1. Nit: The comment should say something of the order that: Reset grace hashjoin context so that there is no state maintained when operator/work is retrieved from cache.

          vikram.dixit Vikram Dixit K added a comment - +1. Nit: The comment should say something of the order that: Reset grace hashjoin context so that there is no state maintained when operator/work is retrieved from cache.
          wzheng Wei Zheng added a comment -

          Thanks for the review. Updated code comment in patch 2.

          wzheng Wei Zheng added a comment - Thanks for the review. Updated code comment in patch 2.

          closeOp() is a dangerous place to put things in, since it can be called during a failure condition.

          This particular reset op should happen in initializeOp().

          gopalv Gopal Vijayaraghavan added a comment - closeOp() is a dangerous place to put things in, since it can be called during a failure condition. This particular reset op should happen in initializeOp().
          wzheng Wei Zheng added a comment -

          patch 3, moving reset logic from closeOp to initializeOp as Gopal suggested.

          wzheng Wei Zheng added a comment - patch 3, moving reset logic from closeOp to initializeOp as Gopal suggested.
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12789353/HIVE-13126.3.patch

          ERROR: -1 due to no test(s) being added or modified.

          ERROR: -1 due to 3 failed/errored test(s), 9803 tests executed
          Failed tests:

          TestMiniTezCliDriver-vector_distinct_2.q-vector_interval_2.q-load_dyn_part2.q-and-12-more - did not produce a TEST-*.xml file
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
          org.apache.hive.jdbc.TestSSL.testSSLVersion
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7075/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7075/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7075/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 3 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12789353 - PreCommit-HIVE-TRUNK-Build

          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12789353/HIVE-13126.3.patch ERROR: -1 due to no test(s) being added or modified. ERROR: -1 due to 3 failed/errored test(s), 9803 tests executed Failed tests: TestMiniTezCliDriver-vector_distinct_2.q-vector_interval_2.q-load_dyn_part2.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7075/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7075/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7075/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12789353 - PreCommit-HIVE-TRUNK-Build
          wzheng Wei Zheng added a comment -

          Vikram Dixit K Can you help commit this patch? Thanks.

          wzheng Wei Zheng added a comment - Vikram Dixit K Can you help commit this patch? Thanks.

          People

            wzheng Wei Zheng Assign to me
            wzheng Wei Zheng
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack