Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1861

Conditional functions with constant arguments should be simplified during analysis

    Details

      Description

      The frontend should be able to detect conditional functions with constant predicates (e.g. IF(true, expr1, expr2)) and automatically replace them with the correct result during the analysis.

        Issue Links

          Activity

          Hide
          caseyc casey added a comment -

          Let's use IMPALA-1788

          Show
          caseyc casey added a comment - Let's use IMPALA-1788
          Hide
          tarmstrong Tim Armstrong added a comment -

          This is a different problem to IMPALA-1788: IF(true, x, y) is not a constant expression so needs a different approach to simplify it.

          Show
          tarmstrong Tim Armstrong added a comment - This is a different problem to IMPALA-1788 : IF(true, x, y) is not a constant expression so needs a different approach to simplify it.
          Hide
          tarmstrong Tim Armstrong added a comment -

          I think this has similar challenges to IMPALA-1788 - will also assign to Alex.

          Show
          tarmstrong Tim Armstrong added a comment - I think this has similar challenges to IMPALA-1788 - will also assign to Alex.
          Hide
          maristir_impala_cf3d Mauricio Aristizabal added a comment -

          Surprised this has been in backlog for 1.5 years. I have a feeling there's many users/queries that would experience a huge performance boost with this 'simple' fix. FYI I opened another jira since marked as dupe of this which has a nice self-contained script to reproduce: IMPALA-4491

          Show
          maristir_impala_cf3d Mauricio Aristizabal added a comment - Surprised this has been in backlog for 1.5 years. I have a feeling there's many users/queries that would experience a huge performance boost with this 'simple' fix. FYI I opened another jira since marked as dupe of this which has a nice self-contained script to reproduce: IMPALA-4491
          Hide
          twmarshall Thomas Tauber-Marshall added a comment -

          commit 4b486b0f90936ab5bc32daf0859bc74b340dd179
          Author: Thomas Tauber-Marshall <tmarshall@cloudera.com>
          Date: Wed Dec 28 15:15:10 2016 -0800

          IMPALA-1861: Simplify conditionals with constant conditions

          When there are conditionals with constant values of TRUE or
          FALSE we can simplify them during analysis using the ExprRewriter.

          This patch introduces the SimplifyConditionalsRule with covers IF,
          OR, AND, CASE, and DECODE.

          It also introduces NormalizeExprsRule which normalizes AND and OR
          such that if either child is a BoolLiteral, then the left child is a
          BoolLiteral.

          Testing:

          • Added unit tests to ExprRewriteRulesTest.
          • Added functional tests to expr.test
          • Ran FE planner tests and BE expr-test.

          Change-Id: Id70aaf9fd99f64bd98175b7e2dbba28f350e7d3b
          Reviewed-on: http://gerrit.cloudera.org:8080/5585
          Reviewed-by: Jim Apple <jbapple-impala@apache.org>
          Tested-by: Impala Public Jenkins

          Show
          twmarshall Thomas Tauber-Marshall added a comment - commit 4b486b0f90936ab5bc32daf0859bc74b340dd179 Author: Thomas Tauber-Marshall <tmarshall@cloudera.com> Date: Wed Dec 28 15:15:10 2016 -0800 IMPALA-1861 : Simplify conditionals with constant conditions When there are conditionals with constant values of TRUE or FALSE we can simplify them during analysis using the ExprRewriter. This patch introduces the SimplifyConditionalsRule with covers IF, OR, AND, CASE, and DECODE. It also introduces NormalizeExprsRule which normalizes AND and OR such that if either child is a BoolLiteral, then the left child is a BoolLiteral. Testing: Added unit tests to ExprRewriteRulesTest. Added functional tests to expr.test Ran FE planner tests and BE expr-test. Change-Id: Id70aaf9fd99f64bd98175b7e2dbba28f350e7d3b Reviewed-on: http://gerrit.cloudera.org:8080/5585 Reviewed-by: Jim Apple <jbapple-impala@apache.org> Tested-by: Impala Public Jenkins
          Hide
          alex.behm Alexander Behm added a comment -

          Unfortunately, this improvement lead to this regression:
          https://issues.apache.org/jira/browse/IMPALA-5125

          Show
          alex.behm Alexander Behm added a comment - Unfortunately, this improvement lead to this regression: https://issues.apache.org/jira/browse/IMPALA-5125
          Hide
          lv Lars Volker added a comment -

          The change to address IMPALA-1861 introduced a regression, which is tracked in IMPALA-5125.

          Show
          lv Lars Volker added a comment - The change to address IMPALA-1861 introduced a regression, which is tracked in IMPALA-5125 .
          Hide
          twmarshall Thomas Tauber-Marshall added a comment -

          See also: IMPALA-5211

          commit 4b486b0f90936ab5bc32daf0859bc74b340dd179
          Author: Thomas Tauber-Marshall <tmarshall@cloudera.com>
          Date: Wed Dec 28 15:15:10 2016 -0800

          IMPALA-1861: Simplify conditionals with constant conditions

          When there are conditionals with constant values of TRUE or
          FALSE we can simplify them during analysis using the ExprRewriter.

          This patch introduces the SimplifyConditionalsRule with covers IF,
          OR, AND, CASE, and DECODE.

          It also introduces NormalizeExprsRule which normalizes AND and OR
          such that if either child is a BoolLiteral, then the left child is a
          BoolLiteral.

          Testing:

          • Added unit tests to ExprRewriteRulesTest.
          • Added functional tests to expr.test
          • Ran FE planner tests and BE expr-test.

          Change-Id: Id70aaf9fd99f64bd98175b7e2dbba28f350e7d3b
          Reviewed-on: http://gerrit.cloudera.org:8080/5585
          Reviewed-by: Jim Apple <jbapple-impala@apache.org>
          Tested-by: Impala Public Jenkins

          Show
          twmarshall Thomas Tauber-Marshall added a comment - See also: IMPALA-5211 commit 4b486b0f90936ab5bc32daf0859bc74b340dd179 Author: Thomas Tauber-Marshall <tmarshall@cloudera.com> Date: Wed Dec 28 15:15:10 2016 -0800 IMPALA-1861 : Simplify conditionals with constant conditions When there are conditionals with constant values of TRUE or FALSE we can simplify them during analysis using the ExprRewriter. This patch introduces the SimplifyConditionalsRule with covers IF, OR, AND, CASE, and DECODE. It also introduces NormalizeExprsRule which normalizes AND and OR such that if either child is a BoolLiteral, then the left child is a BoolLiteral. Testing: Added unit tests to ExprRewriteRulesTest. Added functional tests to expr.test Ran FE planner tests and BE expr-test. Change-Id: Id70aaf9fd99f64bd98175b7e2dbba28f350e7d3b Reviewed-on: http://gerrit.cloudera.org:8080/5585 Reviewed-by: Jim Apple <jbapple-impala@apache.org> Tested-by: Impala Public Jenkins

            People

            • Assignee:
              twmarshall Thomas Tauber-Marshall
              Reporter:
              dtsirogiannis Dimitris Tsirogiannis
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development