[IMPALA-13069] Consider not throwing errors during deleting staging files after insertion

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Backend
• Epic Color: ghx-label-1
• Assignee: Unassigned
• Reporter: Yida Wu
• Created: 11/May/24 05:28
• Updated: 11/May/24 05:28

Description

When inserting data, we often use staging files in the cloud storage as temporary files. Once the insertion is done, we typically clear out these staging files. Currently, our approach involves deleting these files individually and raising errors if any deletion fails. However, at this stage, the insertion process is already complete, and there might be occasional instability with the cloud provider. It may be not optimal to fail a query for such errors during clearing temporary files. Perhaps worth considering a softer way to handle this case.
https://github.com/apache/impala/blob/7ad94006563b88d9221b4ac978dbf5b4fc0a3ca1/be/src/runtime/dml-exec-state.cc#L343