Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:
      None

      Description

      [localhost:21000] > select 1 from (select id, max(int_col) foo from functional.alltypes cc GROUP BY id HAVING (foo < 1) or (foo < 1)) most_recent;
      Query: select 1 from (select id, max(int_col) foo from functional.alltypes cc GROUP BY id HAVING (foo < 1) or (foo < 1)) most_recent
      Query submitted at: 2017-01-06 13:43:11 (Coordinator: http://hnr-optiplex:25000)
      ERROR: AnalysisException: Could not resolve column/field reference: 'foo'
      

      If the query is run with ENABLE_EXPR_REWRITES=0; it completes successfully.

      The bug is sensitive to the structure of the HAVING clause (it's there that the error seems to get thrown from during analysis). If, for example, we have HAVING (foo < 1) or (foo < 2) the query succeeds. However, I've seen a more elaborate reproduction where the disjuncts were not tautological (basically (foo < X) OR (foo < Y) for X < Y).

      Adding extra redundant clauses (like OR FALSE)) often removes the bug as well.

        Activity

        Hide
        marcelk Marcel Kornacker added a comment -

        commit 70ae2e38eb0c4f9be0084e057c70ba427bbbbcfc
        Author: Marcel Kornacker <marcel@cloudera.com>
        Date: Mon Jan 9 18:13:59 2017 -0800

        IMPALA-4739: ExprRewriter fails on HAVING clauses

        The bug was that expr rewrite rules such as ExtractCommonConjunctRule
        analyzed their own output, which doesn't work for syntactic elements
        that allow column aliases, such as the HAVING clause.
        The fix was to remove the analysis step (the re-analysis happens anyway
        in AnalysisCtx).

        Change-Id: Ife74c61f549f620c42f74928f6474e8a5a7b7f00
        Reviewed-on: http://gerrit.cloudera.org:8080/5662
        Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
        Tested-by: Impala Public Jenkins

        Show
        marcelk Marcel Kornacker added a comment - commit 70ae2e38eb0c4f9be0084e057c70ba427bbbbcfc Author: Marcel Kornacker <marcel@cloudera.com> Date: Mon Jan 9 18:13:59 2017 -0800 IMPALA-4739 : ExprRewriter fails on HAVING clauses The bug was that expr rewrite rules such as ExtractCommonConjunctRule analyzed their own output, which doesn't work for syntactic elements that allow column aliases, such as the HAVING clause. The fix was to remove the analysis step (the re-analysis happens anyway in AnalysisCtx). Change-Id: Ife74c61f549f620c42f74928f6474e8a5a7b7f00 Reviewed-on: http://gerrit.cloudera.org:8080/5662 Reviewed-by: Marcel Kornacker <marcel@cloudera.com> Tested-by: Impala Public Jenkins
        Hide
        jbapple Jim Apple added a comment -

        This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09.

        2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that:

        https://github.com/apache/incubator-impala/commits/2.8.0

        Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0?

        Thank you!

        Show
        jbapple Jim Apple added a comment - This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09. 2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that: https://github.com/apache/incubator-impala/commits/2.8.0 Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0? Thank you!

          People

          • Assignee:
            marcelk Marcel Kornacker
            Reporter:
            henryr Henry Robinson
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development