Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11291

Avoid allocation storm while doing rule matching on operator/expression trees

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • None

    Description

      RuleRegExMatch repeatedly allocates string while trying to find a matching pattern. This results in huge GC churn for large trees.

      Attachments

        1. HIVE-11291.patch
          9 kB
          Ashutosh Chauhan

        Issue Links

          Activity

            In most cases, we are trying to do exact matches, regex matching is not required.
            hsubramaniyan Can you take a look?

            ashutoshc Ashutosh Chauhan added a comment - In most cases, we are trying to do exact matches, regex matching is not required. hsubramaniyan Can you take a look?

            ashutoshc The patch looks good to me, +1 pending tests. I believe this idea has to be incorporated as part of HIVE-11141

            Thanks
            Hari

            hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment - ashutoshc The patch looks good to me, +1 pending tests. I believe this idea has to be incorporated as part of HIVE-11141 Thanks Hari
            hiveqa Hive QA added a comment -

            Overall: -1 at least one tests failed

            Here are the results of testing the latest attachment:
            https://issues.apache.org/jira/secure/attachment/12745848/HIVE-11291.patch

            ERROR: -1 due to 2 failed/errored test(s), 9223 tests executed
            Failed tests:

            org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_join0
            org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
            

            Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4640/testReport
            Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4640/console
            Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4640/

            Messages:

            Executing org.apache.hive.ptest.execution.PrepPhase
            Executing org.apache.hive.ptest.execution.ExecutionPhase
            Executing org.apache.hive.ptest.execution.ReportingPhase
            Tests exited with: TestsFailedException: 2 tests failed
            

            This message is automatically generated.

            ATTACHMENT ID: 12745848 - PreCommit-HIVE-TRUNK-Build

            hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12745848/HIVE-11291.patch ERROR: -1 due to 2 failed/errored test(s), 9223 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_join0 org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4640/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4640/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4640/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12745848 - PreCommit-HIVE-TRUNK-Build

            Pushed to master.

            ashutoshc Ashutosh Chauhan added a comment - Pushed to master.

            People

              ashutoshc Ashutosh Chauhan
              ashutoshc Ashutosh Chauhan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: