[SPARK-43046] Implement dropDuplicatesWithinWatermark in Spark Connect

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: Structured Streaming
• Labels: pull-request-available
• Epic Link: Streaming Spark Connect
• Assignee: Unassigned
• Reporter: Jungtaek Lim
• Created: 06/Apr/23 02:00
• Updated: 03/May/24 10:23
• Resolved: 22/Apr/23 00:21

Description

Once SPARK-42931 has merged, we will need to add dropDuplicatesWithinWatermark API to Spark connect, both Python and Scala/Java.

Issue Links

• links to: GitHub Pull Request #46345