Description
Currently in SQL we implement overwrites by calling fs.delete() directly on the original data. This is not ideal since we the original files end up deleted even if the job aborts. We should extend the commit protocol to allow file overwrites to be managed as well.
Attachments
Issue Links
- relates to
-
SPARK-43106 Data lost from the table if the INSERT OVERWRITE query fails
-
- Open
-
- links to
