[SPARK-46265] New assertions in AddArtifact RPC make the connect client incompatible with older clusters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Connect
• Labels: pull-request-available
• Assignee: Niranjan Jayakar
• Reporter: Niranjan Jayakar
• Created: 05/Dec/23 09:36
• Updated: 05/Dec/23 16:44
• Resolved: 05/Dec/23 16:44

Description

We added new assertions to the AddArtifact RPC - https://github.com/apache/spark/commit/d9c5f9d6#diff-d4744c7abd099c57d04746140aba3c20b93f1ac011f5915f963e0a3e0758690e

As part of this change, we have also updated the RPC implementation to return session id as part of its response.

 

Since the assertion depends on the session id to be present, it makes the protocol incompatible such that newer Connect clients that apply this assertion are incompatible with older clusters that don't have the corresponding service side changes.

 

Issue Links

• links to: GitHub Pull Request #44181