Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25182

Block Manager master and slave thread pools are unbounded

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.3.0
    • None
    • Spark Core
    • None

    Description

      Currently, BlockManagerMasterEndpoint and BlockManagerSlaveEndpoint both have thread pools with unbounded numbers of threads. In certain cases, this can lead to driver OOM errors. We should add an upper bound on the number of threads in these thread pools; this should not break any existing behavior because they still have queues of size Integer.MAX_VALUE.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. SPARK-25181 Block Manager master and slave thread pools are unbounded

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              mukulmurthy Mukul Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: