Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2468 Netty-based block server / client module
  3. SPARK-3503

Disable thread local cache in PooledByteBufAllocator

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersConvert to IssueLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: Shuffle, Spark Core
    • Labels:
      None
    • Target Version/s:

      Description

      Feedback from Norman.

      Thread-local caches should be disabled because the ByteBufs are allocated by the event loop thread, but released by the executor thread rather than the event loop thread. Those thread-local caches actually delay the recycling of buffers, leading to larger memory usage.

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment