Description
In https://github.com/apache/spark/pull/48320 the `ArtifactManager.clone` method, we re-register all existing artifacts with new Session UUID to the new SparkContext. This current logic is a bit heavy and unnecessary because all artifacts are already in place and have passed all necessary checks.
Ideally, we want to modify the SparkContext.addedJars map directly to insert these artifacts with new session UUIDs - or something that is lightweight.
Attachments
Issue Links
- depends upon
-
SPARK-49854 Clone artifact manager during sparkSession.cloneSession
-
- Open
-