[SPARK-28551] CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.2.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Vinod KC
• Reporter: angerszhu
• Created: 29/Jul/19 11:10
• Updated: 16/Jan/24 03:45
• Resolved: 20/May/21 06:13

Description

When we run SQL like 

CRETE TABLE TBL 
LOCATION 'PATH_URI'
AS
SELECT QUERY

It won't check PATH_URI status, if there is some data under this path, it will just overwrite this path.

If user careless write a path with important data store under the path, it will be a disaster。 

Thought about add a checker in Catalyst for this situation.

In addition, if it fails to delete the old path completely, old data and new data coexists.

Issue Links

• links to: [Github] Pull Request #32411 (vinodkc)
• links to: [Github] Pull Request #32411 (vinodkc)
• links to: [Github] Pull Request #32618 (cloud-fan)
• links to: GitHub Pull Request #25290
• links to: GitHub Pull Request #32411