[IMPALA-2821] error when trying to drop or compute stats of table with a nested field in a struct column named `timestamp`

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.3.0
• Component/s: Frontend
• Assignee: Unassigned
• Reporter: oleksii iepishkin
• Created: 07/Jan/16 18:55
• Updated: 20/Jan/16 20:05
• Resolved: 20/Jan/16 20:05

Description

Issue: quoted identifiers don't work in structs in external tables. I can create a table but can't drop it or compute stats. The error is

ERROR: AnalysisException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'

CDH 5.5.1 with Impala v2.3.0-cdh5.5.1 (73bf5bc) built on Wed Dec 2 10:39:33 PST 2015

How to reproduce: (spark-shell on one of datanodes)
This works

[localhost:21000] > create external table test4 (header struct<`ts`: bigint>) stored as parquet;
Query: create external table test4 (header struct<`ts`: bigint>) stored as parquet
Fetched 0 row(s) in 0.08s

[localhost:21000] > compute stats test4;
Query: compute stats test4
+-----------------------------------------+
| summary                                 |
+-----------------------------------------+
| Updated 1 partition(s) and 0 column(s). |
+-----------------------------------------+
Fetched 1 row(s) in 6.68s

[localhost:21000] > drop table test4;
Query: drop table test4

This fails

[localhost:21000] > create external table test3 (header struct<`timestamp`: bigint>) stored as parquet;
Query: create external table test3 (header struct<`timestamp`: bigint>) stored as parquet
Fetched 0 row(s) in 0.06s

[localhost:21000] > drop table test3;
Query: drop table test3
ERROR: AnalysisException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'
CAUSED BY: TableLoadingException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'

Compute stats fails too

[localhost:21000] > compute stats test3;
Query: compute stats test3
ERROR: AnalysisException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'
CAUSED BY: TableLoadingException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'

Can not query this table

[localhost:21000] > select header.`timestamp` from test3;
Query: select header.`timestamp` from test3
ERROR: AnalysisException: Failed to load metadata for table: 'test3'
CAUSED BY: TableLoadingException: Unsupported type 'struct<timestamp:bigint>' in column 'header' of table 'test3'

Top level columns support quoted identifiers

[localhost:21000] > create external table test2 (`timestamp` bigint) stored as parquet;
Query: create external table test2 (`timestamp` bigint) stored as parquet
Fetched 0 row(s) in 0.09s

[localhost:21000] > compute stats test2;
Query: compute stats test2
+-----------------------------------------+
| summary                                 |
+-----------------------------------------+
| Updated 1 partition(s) and 1 column(s). |
+-----------------------------------------+
Fetched 1 row(s) in 4.54s

[localhost:21000] > drop table test2;
Query: drop table test3

Workaround found for table removal: drop the table from hive metastore in HUE.

No workaround found for compute stats or querying the table

Issue Links

• duplicates: IMPALA-2820 Impala does not support reserved words in complex types