Hive
  1. Hive
  2. HIVE-4160 Vectorized Query Execution in Hive
  3. HIVE-5604

Fix validation of nested expressions.

        Details

        • Type: Sub-task Sub-task
        • Status: Resolved
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 0.13.0
        • Component/s: Vectorization
        • Labels:
          None

          Description

          The bug fixes a few issues related to nested expressions:
          1) The nested expressions were not being validated at all.
          2) UDFRegExp was not handled correctly, but issue was not caught because of the previous issue.
          3) HIVE-5642 will not show up when this jira is fixed, but still added a sanity check to validate the number of arguments.

          1. Text File
            HIVE-5604.1.patch
            6 kB
            Jitendra Nath Pandey
          2. Text File
            HIVE-5604.2.patch
            12 kB
            Jitendra Nath Pandey
          3. Text File
            HIVE-5604.3.patch
            18 kB
            Jitendra Nath Pandey
          4. Text File
            HIVE-5604.4.patch
            19 kB
            Jitendra Nath Pandey
          5. Text File
            HIVE-5604.tez.patch
            18 kB
            Jitendra Nath Pandey

            Issue Links

            incorporates

            Sub-task - The sub-task of the issue HIVE-5642 Exception in UDFs with large number of arguments.

            • Major - Major loss of function.
            • Resolved
            is blocked by

            Sub-task - The sub-task of the issue HIVE-5430 Refactor VectorizationContext and handle NOT expression with nulls.

            • Major - Major loss of function.
            • Resolved

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Jitendra Nath Pandey added a comment -

              Attached patch. But patch will apply only after HIVE-5430 is committed.

              Show
              Jitendra Nath Pandey added a comment - Attached patch. But patch will apply only after HIVE-5430 is committed.
              Hide
              Permalink
              Hive QA added a comment -

              Overall: -1 at least one tests failed

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12609952/HIVE-5604.2.patch

              ERROR: -1 due to 3 failed/errored test(s), 4471 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/console

              Messages:

              Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 3 tests failed

              This message is automatically generated.

              Show
              Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12609952/HIVE-5604.2.patch ERROR: -1 due to 3 failed/errored test(s), 4471 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 3 tests failed This message is automatically generated.
              Hide
              Permalink
              Ashutosh Chauhan added a comment -

              Can you create a phabricator / review board entry? Also, can you describe a bug a bit.

              Show
              Ashutosh Chauhan added a comment - Can you create a phabricator / review board entry? Also, can you describe a bug a bit.
              Hide
              Permalink
              Jitendra Nath Pandey added a comment -

              Review board entry: https://reviews.apache.org/r/14956/

              Show
              Jitendra Nath Pandey added a comment - Review board entry: https://reviews.apache.org/r/14956/
              Hide
              Permalink
              Jitendra Nath Pandey added a comment -

              Also, updated the description. The crux of the issue was that children expressions were not being validated and that was masking some other issues as well.

              Show
              Jitendra Nath Pandey added a comment - Also, updated the description. The crux of the issue was that children expressions were not being validated and that was masking some other issues as well.
              Hide
              Permalink
              Ashutosh Chauhan added a comment -

              Comment for Builder class of VectorExprDesc is still valid, right? Looks like you inadvertently removed it.

              Show
              Ashutosh Chauhan added a comment - Comment for Builder class of VectorExprDesc is still valid, right? Looks like you inadvertently removed it.
              Hide
              Permalink
              Jitendra Nath Pandey added a comment -

              The comment was removed because the descriptor is not a bit vector anymore, instead it is an immutable object. I have updated the patch to add additional comments.

              Show
              Jitendra Nath Pandey added a comment - The comment was removed because the descriptor is not a bit vector anymore, instead it is an immutable object. I have updated the patch to add additional comments.
              Hide
              Permalink
              Ashutosh Chauhan added a comment -

              +1

              Show
              Ashutosh Chauhan added a comment - +1
              Hide
              Permalink
              Hive QA added a comment -

              Overall: +1 all checks pass

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12610697/HIVE-5604.4.patch

              SUCCESS: +1 4513 tests passed

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/console

              Messages:

              Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase

              This message is automatically generated.

              Show
              Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610697/HIVE-5604.4.patch SUCCESS: +1 4513 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
              Hide
              Permalink
              Ashutosh Chauhan added a comment -

              Committed to trunk. Thanks, Jitendra!

              Show
              Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Jitendra!

                People

                • Assignee:
                  Jitendra Nath Pandey
                  Reporter:
                  Jitendra Nath Pandey
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development