Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37063

SQL Adaptive Query Execution QA: Phase 2

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.0
    • None
    • SQL
    • None

    Attachments

      Issue Links

        1.
        Support fetching shuffle blocks in batch with i/o encryption Sub-task Reopened Unassigned
        2.
        Completely fix the broadcast timeout issue in AQE Sub-task Resolved Unassigned
        3.
        Flaky Test: AdaptiveQueryExecSuite.multiple joins Sub-task Resolved Unassigned
        4.
        Make config maxShuffledHashJoinLocalMapThreshold fallback to advisoryPartitionSizeInBytes Sub-task In Progress Unassigned
        5.
        Demote BroadcastJoin causes performance regression and increases OOM risks Sub-task Open Unassigned
        6.
        Cancel all running job after AQE plan finished Sub-task In Progress Unassigned
        7.
        OptimizeSkewInRebalancePartitions support optimize non-root node Sub-task Resolved XiDuo You
        8.
        Add small partition factor for rebalance partitions Sub-task Resolved XiDuo You
        9.
        Support optimize skewed join through union Sub-task Resolved Unassigned
        10.
        In AQE, wrong InMemoryRelation size estimation causes "Cannot broadcast the table that is larger than 8GB: 8 GB" failure Sub-task Resolved Michael Chen
        11.
        SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on whole plan innstead of new stage plan Sub-task Resolved Wenchen Fan
        12.
        Add an option in AQE to mitigate skew even if it causes an new shuffle Sub-task Resolved XiDuo You
        13.
        Support coalesce partition through union Sub-task Resolved XiDuo You
        14.
        Support eliminate limits in AQE Optimizer Sub-task Resolved XiDuo You
        15.
        AQE dynamic join selection should not apply to non-equi join Sub-task Resolved Cheng Su
        16.
        AQE can change bhj to smj if no extra shuffle introduce Sub-task Resolved XiDuo You
        17.
        Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1 Sub-task Resolved Shardul Mahadik
        18.
        Explain formatted does not collect subqueries under query stage in AQE Sub-task Resolved XiDuo You
        19.
        Unify get preferred locations for shuffle in AQE Sub-task In Progress Unassigned
        20.
        Improve perfermance of ShufflePartitionsUtil createSkewPartitionSpecs Sub-task Resolved XiDuo You
        21.
        AQE throw exception should respect SparkFatalException Sub-task Resolved XiDuo You
        22.
        TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false Sub-task Resolved XiDuo You
        23.
        Avoid unnecessary sort in FileFormatWriter if user has specified sort in AQE Sub-task In Progress Unassigned
        24.
        Extend SparkSessionExtensions to inject rules into AQE Optimizer Sub-task Resolved XiDuo You
        25.
        Optimize one row plan in normal and AQE Optimizer Sub-task Resolved Unassigned

        Activity

          People

            Unassigned Unassigned
            dongjoon Dongjoon Hyun
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: