Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4160 Vectorized Query Execution in Hive
  3. HIVE-5604

Fix validation of nested expressions.

        Details

        • Type: Sub-task
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 0.13.0
        • Component/s: Vectorization
        • Labels:
          None

          Description

          The bug fixes a few issues related to nested expressions:
          1) The nested expressions were not being validated at all.
          2) UDFRegExp was not handled correctly, but issue was not caught because of the previous issue.
          3) HIVE-5642 will not show up when this jira is fixed, but still added a sanity check to validate the number of arguments.

          1. Text File
            HIVE-5604.1.patch
            6 kB
            Jitendra Nath Pandey
          2. Text File
            HIVE-5604.2.patch
            12 kB
            Jitendra Nath Pandey
          3. Text File
            HIVE-5604.3.patch
            18 kB
            Jitendra Nath Pandey
          4. Text File
            HIVE-5604.4.patch
            19 kB
            Jitendra Nath Pandey
          5. Text File
            HIVE-5604.tez.patch
            18 kB
            Jitendra Nath Pandey

            Issue Links

            incorporates

            Sub-task - The sub-task of the issue HIVE-5642 Exception in UDFs with large number of arguments.

            • Major - Major loss of function.
            • Resolved
            is blocked by

            Sub-task - The sub-task of the issue HIVE-5430 Refactor VectorizationContext and handle NOT expression with nulls.

            • Major - Major loss of function.
            • Resolved

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              jnp Jitendra Nath Pandey added a comment -

              Attached patch. But patch will apply only after HIVE-5430 is committed.

              Show
              jnp Jitendra Nath Pandey added a comment - Attached patch. But patch will apply only after HIVE-5430 is committed.
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Overall: -1 at least one tests failed

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12609952/HIVE-5604.2.patch

              ERROR: -1 due to 3 failed/errored test(s), 4471 tests executed
              Failed tests:

              org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/console

              Messages:

              Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 3 tests failed

              This message is automatically generated.

              Show
              hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12609952/HIVE-5604.2.patch ERROR: -1 due to 3 failed/errored test(s), 4471 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1227/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 3 tests failed This message is automatically generated.
              Hide
              Permalink
              ashutoshc Ashutosh Chauhan added a comment -

              Can you create a phabricator / review board entry? Also, can you describe a bug a bit.

              Show
              ashutoshc Ashutosh Chauhan added a comment - Can you create a phabricator / review board entry? Also, can you describe a bug a bit.
              Hide
              Permalink
              jnp Jitendra Nath Pandey added a comment -

              Review board entry: https://reviews.apache.org/r/14956/

              Show
              jnp Jitendra Nath Pandey added a comment - Review board entry: https://reviews.apache.org/r/14956/
              Hide
              Permalink
              jnp Jitendra Nath Pandey added a comment -

              Also, updated the description. The crux of the issue was that children expressions were not being validated and that was masking some other issues as well.

              Show
              jnp Jitendra Nath Pandey added a comment - Also, updated the description. The crux of the issue was that children expressions were not being validated and that was masking some other issues as well.
              Hide
              Permalink
              ashutoshc Ashutosh Chauhan added a comment -

              Comment for Builder class of VectorExprDesc is still valid, right? Looks like you inadvertently removed it.

              Show
              ashutoshc Ashutosh Chauhan added a comment - Comment for Builder class of VectorExprDesc is still valid, right? Looks like you inadvertently removed it.
              Hide
              Permalink
              jnp Jitendra Nath Pandey added a comment -

              The comment was removed because the descriptor is not a bit vector anymore, instead it is an immutable object. I have updated the patch to add additional comments.

              Show
              jnp Jitendra Nath Pandey added a comment - The comment was removed because the descriptor is not a bit vector anymore, instead it is an immutable object. I have updated the patch to add additional comments.
              Hide
              Permalink
              ashutoshc Ashutosh Chauhan added a comment -

              +1

              Show
              ashutoshc Ashutosh Chauhan added a comment - +1
              Hide
              Permalink
              hiveqa Hive QA added a comment -

              Overall: +1 all checks pass

              Here are the results of testing the latest attachment:
              https://issues.apache.org/jira/secure/attachment/12610697/HIVE-5604.4.patch

              SUCCESS: +1 4513 tests passed

              Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/testReport
              Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/console

              Messages:

              Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase

              This message is automatically generated.

              Show
              hiveqa Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610697/HIVE-5604.4.patch SUCCESS: +1 4513 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1282/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
              Hide
              Permalink
              ashutoshc Ashutosh Chauhan added a comment -

              Committed to trunk. Thanks, Jitendra!

              Show
              ashutoshc Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Jitendra!

                People

                • Assignee:
                  jnp Jitendra Nath Pandey
                  Reporter:
                  jnp Jitendra Nath Pandey
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development