Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10783

Do track the pointer array in UnsafeInMemorySorter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.5.1, 1.6.0
    • 1.6.0
    • SQL
    • None

    Description

      SPARK-10474 (https://github.com/apache/spark/pull/8888) removed the pointer array tracking because `TungstenAggregate` would fail under memory pressure. However, this is somewhat of a hack that we should fix in the right way in 1.6.0 to ensure we don't OOM because of this.

      Attachments

        Issue Links

          Activity

            People

              davies Davies Liu
              andrewor14 Andrew Or
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: