Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26633

Add ExecutorClassLoader.getResourceAsStream

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.4.1, 3.0.0
    • Component/s: Spark Shell
    • Labels:
      None

      Description

      ExecutorClassLoader is capable of loading dynamically generated classes from the REPL via either RPC or HDFS, but right now it always delegates resource loading to the parent class loader. That makes the dynamically generated classes unavailable to uses other than class loading.

      Such need may arise, for example, when json4s wants to parse the Class file to extract parameter name information. Internally it'd call the class loader's getResourceAsStream to obtain the Class file content as an InputStream.

      This ticket tracks an improvement to the ExecutorClassLoader to allow fetching dynamically generated Class files from the REPL as resource streams.

        Attachments

          Activity

            People

            • Assignee:
              rednaxelafx Kris Mok
              Reporter:
              rednaxelafx Kris Mok
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: