Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.13.1
    • 0.14.0
    • Statistics
    • None

    Description

      When StatsRulesProcFactory overflows it sets data size to 0 and as a result the Vertex will ask for a single task, this results in a fairly slow running query, most likely the overflow is a result of higher than usual number of rows.

      The class should detect an overflow and set a flag when an overflow occurs, if an overflow occurs StatsRulesProcFactory should request the maximum number of tasks for the vertex.

      Attachments

        1. HIVE-7992.1.patch
          17 kB
          Prasanth Jayachandran

        Issue Links

          Activity

            rhbutani Harish Butani added a comment -

            +1

            rhbutani Harish Butani added a comment - +1
            hiveqa Hive QA added a comment -

            Overall: -1 at least one tests failed

            Here are the results of testing the latest attachment:
            https://issues.apache.org/jira/secure/attachment/12666953/HIVE-7992.1.patch

            ERROR: -1 due to 3 failed/errored test(s), 6184 tests executed
            Failed tests:

            org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
            org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
            org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
            

            Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/671/testReport
            Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/671/console
            Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-671/

            Messages:

            Executing org.apache.hive.ptest.execution.PrepPhase
            Executing org.apache.hive.ptest.execution.ExecutionPhase
            Executing org.apache.hive.ptest.execution.ReportingPhase
            Tests exited with: TestsFailedException: 3 tests failed
            

            This message is automatically generated.

            ATTACHMENT ID: 12666953

            hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12666953/HIVE-7992.1.patch ERROR: -1 due to 3 failed/errored test(s), 6184 tests executed Failed tests: org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/671/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/671/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-671/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12666953

            Committed to trunk. Thanks Harish for the review.

            prasanth_j Prasanth Jayachandran added a comment - Committed to trunk. Thanks Harish for the review.
            thejas Thejas Nair added a comment -

            This has been fixed in 0.14 release. Please open new jira if you see any issues.

            thejas Thejas Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

              prasanth_j Prasanth Jayachandran
              mmokhtar Mostafa Mokhtar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: