Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35791

Release on-going map properly for NULL-aware ANTI join

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0, 3.1.3
    • SQL
    • None

    Description

      NULL-aware ANTI join (https://issues.apache.org/jira/browse/SPARK-32290) detects NULL join keys during building the map for `HashedRelation`, and will immediately return `HashedRelationWithAllNullKeys` without taking care of the map built already. Before returning `HashedRelationWithAllNullKeys`, the map needs to be freed properly to save memory and keep memory accounting correctly.

       

      Attachments

        Activity

          apachespark Apache Spark added a comment -

          User 'c21' has created a pull request for this issue:
          https://github.com/apache/spark/pull/32939

          apachespark Apache Spark added a comment - User 'c21' has created a pull request for this issue: https://github.com/apache/spark/pull/32939
          apachespark Apache Spark added a comment -

          User 'c21' has created a pull request for this issue:
          https://github.com/apache/spark/pull/32939

          apachespark Apache Spark added a comment - User 'c21' has created a pull request for this issue: https://github.com/apache/spark/pull/32939
          cloud_fan Wenchen Fan added a comment -

          Issue resolved by pull request 32939
          https://github.com/apache/spark/pull/32939

          cloud_fan Wenchen Fan added a comment - Issue resolved by pull request 32939 https://github.com/apache/spark/pull/32939

          People

            chengsu Cheng Su
            chengsu Cheng Su
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: