[SPARK-49050] Enabling deleteIfExists operator in TWS with Virtual Column Families

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Structured Streaming
• Labels: pull-request-available
• Epic Link: Structured Streaming - Arbitrary State API v2
• Assignee: Eric Marnadi
• Reporter: Eric Marnadi
• Created: 29/Jul/24 21:17
• Updated: 28/Aug/24 06:41
• Resolved: 28/Aug/24 06:41

Description

Fully integrating the TransformWithState operator with Virtual Column Families by adding support for the deleteIfExists operator. In RocksDB.scala, return the columnFamilyId if it exists in removeColFamilyIfExists so that we can remove all the keys for the corresponding columnFamily once deleteIfExists is called.

Issue Links

• links to: GitHub Pull Request #47524
• links to: GitHub Pull Request #47880