Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21956

Fetch up to max bytes when buf really released

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • 2.1.0
    • None
    • Shuffle, Spark Core
    • None

    Description

      Right now,ShuffleBlockFetcherIterator will decrease bytesInFlight immediatily when take from results queue.But currentresult's bytebuf has not been released at that time.Then direct memory may be a little out of control.
      We should decrease bytesInFlight when currentresult's bytebuf really been released.

      Attachments

        Activity

          People

            Unassigned Unassigned
            cane zhoukang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: