Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35553

Improve correlated subqueries

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.2.0, 3.3.0, 3.4.0
    • None
    • SQL
    • None

    Description

      This umbrella ticket is used to keep track of correlated subquery improvements and bug fixes. 

      The goal is to gradually remove the restrictions in CheckAnalysis for correlated subqueries.

      Attachments

        1.
        Improve the subquery decorrelation framework Sub-task Resolved Allison Wang
        2.
        Support outer references in Aggregate Sub-task Open Unassigned
        3.
        Support subqueries with correlated non-equality predicates Sub-task Resolved Allison Wang
        4.
        Support set operators to be on correlation paths Sub-task In Progress Unassigned
        5.
        Support ORDER BY and LIMIT to be on the correlation path Sub-task Open Unassigned
        6.
        Allow Project to host outer references in scalar subqueries Sub-task Resolved Allison Wang
        7.
        Optimize OneRowRelation subqueries Sub-task Resolved Allison Wang
        8.
        Enable DecorrelateInnerQuery for IN/EXISTS subqueries Sub-task Open Unassigned
        9.
        Unify the logic to handle COUNT bug for scalar and lateral subqueries Sub-task Open Unassigned
        10.
        Handle the COUNT bug for correlated IN/EXISTS subquery Sub-task Open Unassigned
        11.
        ResolveSubquery should skip unary node UnresolvedHaving Sub-task Open Unassigned
        12.
        Join can become unresolved after PullupCorrelatedPredicates Sub-task Open Unassigned
        13.
        CollapseProject should not collapse correlated scalar subqueries Sub-task Resolved Allison Wang
        14.
        Do not collapse Project with Aggregate when correlated subqueries are present in the project list Sub-task Resolved Allison Wang
        15.
        Disallow distinct aggregate in lateral subqueries with unsupported correlated predicates Sub-task Resolved Allison Wang
        16.
        Allow safe up-cast expressions in correlated equality predicates Sub-task Resolved Allison Wang
        17.
        Check unsupported data type when decorrelating subqueries Sub-task Resolved Allison Wang
        18.
        Refactor checkCorrelationsInSubquery Sub-task Resolved Allison Wang
        19.
        Always inline expressions in OptimizeOneRowRelationSubquery Sub-task Resolved Allison Wang
        20.
        Unexpected operators when rewriting scalar subqueries with non-deterministic expressions Sub-task Resolved Allison Wang

        Activity

          People

            Unassigned Unassigned
            allisonwang-db Allison Wang
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: