[IMPALA-3287] S3 : Create table like with S3 source table creates a new table with HDFS as location

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Labels: s3
• Target Version: Impala 2.6.0
• Assignee: Sailesh Mukil
• Reporter: Mostafa Mokhtar
• Created: 31/Mar/16 20:50
• Updated: 02/May/16 22:28
• Resolved: 02/May/16 22:28

Description

Source table

[impala-s3-r3-2.vpc.cloudera.com:21000] > describe formatted lineitem;
Query: describe formatted lineitem
+------------------------------+----------------------------------------------------------------+----------------------+
| name                         | type                                                           | comment              |
+------------------------------+----------------------------------------------------------------+----------------------+
| # col_name                   | data_type                                                      | comment              |
|                              | NULL                                                           | NULL                 |
| l_orderkey                   | bigint                                                         | NULL                 |
| l_partkey                    | bigint                                                         | NULL                 |
| l_suppkey                    | bigint                                                         | NULL                 |
| l_linenumber                 | bigint                                                         | NULL                 |
| l_quantity                   | decimal(12,2)                                                  | NULL                 |
| l_extendedprice              | decimal(12,2)                                                  | NULL                 |
| l_discount                   | decimal(12,2)                                                  | NULL                 |
| l_tax                        | decimal(12,2)                                                  | NULL                 |
| l_returnflag                 | string                                                         | NULL                 |
| l_linestatus                 | string                                                         | NULL                 |
| l_shipdate                   | string                                                         | NULL                 |
| l_commitdate                 | string                                                         | NULL                 |
| l_receiptdate                | string                                                         | NULL                 |
| l_shipinstruct               | string                                                         | NULL                 |
| l_shipmode                   | string                                                         | NULL                 |
| l_comment                    | string                                                         | NULL                 |
|                              | NULL                                                           | NULL                 |
| # Detailed Table Information | NULL                                                           | NULL                 |
| Database:                    | tpch_300_parquet_s3                                            | NULL                 |
| Owner:                       | root                                                           | NULL                 |
| CreateTime:                  | Wed Mar 23 17:01:32 PDT 2016                                   | NULL                 |
| LastAccessTime:              | UNKNOWN                                                        | NULL                 |
| Protect Mode:                | None                                                           | NULL                 |
| Retention:                   | 0                                                              | NULL                 |
| Location:                    | s3a://XXXXXXXX/tpch_300_parquet.db/lineitem     | NULL                 |
| Table Type:                  | EXTERNAL_TABLE                                                 | NULL                 |
| Table Parameters:            | NULL                                                           | NULL                 |
|                              | COLUMN_STATS_ACCURATE                                          | true                 |
|                              | EXTERNAL                                                       | TRUE                 |
|                              | numFiles                                                       | 0                    |
|                              | numRows                                                        | 1799989091           |
|                              | rawDataSize                                                    | -1                   |
|                              | totalSize                                                      | 0                    |
|                              | transient_lastDdlTime                                          | 1458944599           |
|                              | NULL                                                           | NULL                 |
| # Storage Information        | NULL                                                           | NULL                 |
| SerDe Library:               | org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe    | NULL                 |
| InputFormat:                 | org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat  | NULL                 |
| OutputFormat:                | org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat | NULL                 |
| Compressed:                  | No                                                             | NULL                 |
| Num Buckets:                 | 0                                                              | NULL                 |
| Bucket Columns:              | []                                                             | NULL                 |
| Sort Columns:                | []                                                             | NULL                 |
+------------------------------+----------------------------------------------------------------+----------------------+

DDL

 create table lineitem_s3_parquet like lineitem;

Created table

Query: describe formatted lineitem_s3_parquet
+------------------------------+------------------------------------------------------------------------------------------------------------+----------------------+
| name                         | type                                                                                                       | comment              |
+------------------------------+------------------------------------------------------------------------------------------------------------+----------------------+
| # col_name                   | data_type                                                                                                  | comment              |
|                              | NULL                                                                                                       | NULL                 |
| l_orderkey                   | bigint                                                                                                     | NULL                 |
| l_partkey                    | bigint                                                                                                     | NULL                 |
| l_suppkey                    | bigint                                                                                                     | NULL                 |
| l_linenumber                 | bigint                                                                                                     | NULL                 |
| l_quantity                   | decimal(12,2)                                                                                              | NULL                 |
| l_extendedprice              | decimal(12,2)                                                                                              | NULL                 |
| l_discount                   | decimal(12,2)                                                                                              | NULL                 |
| l_tax                        | decimal(12,2)                                                                                              | NULL                 |
| l_returnflag                 | string                                                                                                     | NULL                 |
| l_linestatus                 | string                                                                                                     | NULL                 |
| l_shipdate                   | string                                                                                                     | NULL                 |
| l_commitdate                 | string                                                                                                     | NULL                 |
| l_receiptdate                | string                                                                                                     | NULL                 |
| l_shipinstruct               | string                                                                                                     | NULL                 |
| l_shipmode                   | string                                                                                                     | NULL                 |
| l_comment                    | string                                                                                                     | NULL                 |
|                              | NULL                                                                                                       | NULL                 |
| # Detailed Table Information | NULL                                                                                                       | NULL                 |
| Database:                    | tpch_300_parquet_s3                                                                                        | NULL                 |
| Owner:                       | root                                                                                                       | NULL                 |
| CreateTime:                  | Thu Mar 31 13:38:05 PDT 2016                                                                               | NULL                 |
| LastAccessTime:              | UNKNOWN                                                                                                    | NULL                 |
| Protect Mode:                | None                                                                                                       | NULL                 |
| Retention:                   | 0                                                                                                          | NULL                 |
| Location:                    | hdfs://impala-s3-r3-1.vpc.cloudera.com:8020/user/hive/warehouse/tpch_300_parquet_s3.db/lineitem_s3_parquet | NULL                 |
| Table Type:                  | MANAGED_TABLE                                                                                              | NULL                 |
| Table Parameters:            | NULL                                                                                                       | NULL                 |
|                              | COLUMN_STATS_ACCURATE                                                                                      | true                 |
|                              | numFiles                                                                                                   | 0                    |
|                              | numRows                                                                                                    | -1                   |
|                              | rawDataSize                                                                                                | -1                   |
|                              | totalSize                                                                                                  | 0                    |
|                              | transient_lastDdlTime                                                                                      | 1458944599           |
|                              | NULL                                                                                                       | NULL                 |
| # Storage Information        | NULL                                                                                                       | NULL                 |
| SerDe Library:               | org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe                                                | NULL                 |
| InputFormat:                 | org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat                                              | NULL                 |
| OutputFormat:                | org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat                                             | NULL                 |
| Compressed:                  | No                                                                                                         | NULL                 |
| Num Buckets:                 | 0                                                                                                          | NULL                 |
| Bucket Columns:              | []                                                                                                         | NULL                 |
| Sort Columns:                | []                                                                                                         | NULL                 |
+------------------------------+------------------------------------------------------------------------------------------------------------+----------------------+