Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24225

Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.6.3, 2.2.0, 2.2.1, 2.3.0
    • Fix Version/s: None
    • Component/s: Block Manager, Spark Core
    • Labels:
      None

      Description

      When using Broadcast Variables, it would be beneficial if classes implementing AutoClosable were closed when released. This would allow broadcast variables to be used, for example, as shared resource pools across multiple tasks within an executor without the use of static variables. 

        Attachments

          Activity

            People

            • Assignee:
              drohrer Doug Rohrer
              Reporter:
              drohrer Doug Rohrer
              Shepherd:
              DB Tsai
            • Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: