[DRILL-4896] After a failed CTAS, the table both exists and does not exist

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.8.0
• Component/s: Server
• Assignee: Unassigned
• Reporter: Boaz Ben-Zvi
• Created: 20/Sep/16 01:29
• Updated: 14/Sep/18 13:39

Description

After CTAS failed (due to no space on storage device) there were (incomplete) Parquet files left. A subsequent CTAS for the same table name fails with "table exists", and a subsequent DROP on the same table name fails with "table does not exist".

A possible enhancement: DROP to be able to cleanup such a corrupted table.

0: jdbc:drill:zk=local> create table `/drill/spill/tt1` as
. . . . . . . . . . . > select
. . . . . . . . . . . > case when columns[2] = '' then cast(null as varchar(100)) else cast(columns[2] as varchar(100)) end,
. . . . . . . . . . . > case when columns[3] = '' then cast(null as varchar(100)) else cast(columns[3] as varchar(100)) end,
. . . . . . . . . . . > case when columns[4] = '' then cast(null as varchar(100)) else cast(columns[4] as varchar(100)) end,
. . . . . . . . . . . > case when columns[5] = '' then cast(null as varchar(100)) else cast(columns[5] as varchar(100)) end,
. . . . . . . . . . . > case when columns[0] = '' then cast(null as varchar(100)) else cast(columns[0] as varchar(100)) end,
. . . . . . . . . . . > case when columns[8] = '' then cast(null as varchar(100)) else cast(columns[8] as varchar(100)) end
. . . . . . . . . . . > FROM dfs.`/Users/boazben-zvi/data/store_sales/store_sales.dat`;
Exception in thread "drill-executor-4" org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
......... 39 more
Error: SYSTEM ERROR: IOException: The file being written is in an invalid state. Probably caused by an error thrown previously. Current state: COLUMN

Fragment 0:0

[Error Id: de84c212-2400-4a08-a15c-8e3adb5ec774 on 10.250.57.63:31010] (state=,code=0)
0: jdbc:drill:zk=local> create table `/drill/spill/tt1` as select * from dfs.`/Users/boazben-zvi/data/store_sales/store_sales.dat`;
Error: VALIDATION ERROR: A table or view with given name [/drill/spill/tt1] already exists in schema [dfs.tmp]

[Error Id: 0ef99a15-9d67-49ad-87fb-023105dece3c on 10.250.57.63:31010] (state=,code=0)
0: jdbc:drill:zk=local> drop table `/drill/spill/tt1` ;
Error: DATA_WRITE ERROR: Failed to drop table: File /drill/spill/tt1 does not exist

[Error Id: c22da79f-ecbd-423c-b5b2-4eae7d1263d7 on 10.250.57.63:31010] (state=,code=0)