[SPARK-44476] JobArtifactSet is populated with all artifacts if it is not associated with an artifact

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0, 4.0.0
• Fix Version/s: 3.5.0, 4.0.0
• Component/s: Connect
• Epic Link: Spark Connect Scala Client - Basic
• Assignee: Venkata Sai Akhil Gudesa
• Reporter: Venkata Sai Akhil Gudesa
• Created: 18/Jul/23 14:34
• Updated: 19/Jul/23 00:37
• Resolved: 19/Jul/23 00:37

Description

Consider each artifact type - files/jars/archives. For each artifact type, the following bug exists:

1. Initialise a `JobArtifactState` with no artifacts added to it.
2. Create a  `JobArtifactSet` from the `JobArtifactState`.
3. Add an artifact with the same active `JobArtifactState`.
4. Create another `JobArtifactSet`

In the current behaviour, the set created in step 2 contains all the artifacts (through `sc.allAddedFiles` for example) while step 3 contains only the single artifact added in step 3.