Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 1.1.1
-
None
-
None
Description
I started with a text table structured like so:
------------------------
| name | type | comment |
------------------------
| id | int | |
| val | int | |
| zfill | string | |
| name | string | |
| assertion | string | |
| year | string | |
| month | string | |
| day | string |
------------------------
I gathered column stats via Hive ANALYZE TABLE for the year, month, and day columns.
Back in Impala, I ran ALTER TABLE REPLACE COLUMNS to switch some columns (including the ones with stats) away from STRING:
alter table raw_data_ymd replace columns (id int, val int, zfill int, name string, assertion boolean, year int, month int, day int);
After that, any attempt to access the table fails:
[localhost:21000] > desc raw_data_ymd;
Query: describe raw_data_ymd
ERROR: AnalysisException: Failed to load metadata for table: partitions.raw_data_ymd
CAUSED BY: TableLoadingException: Failed to load metadata for table: raw_data_ymd
CAUSED BY: IllegalStateException: null
INVALIDATE METADATA doesn't help.
The table can still be DESCRIBEd and queried in Hive.
The Impala log shows this detail:
I0910 15:11:18.969578 11401 Frontend.java:338] analyze query select count
from raw_data_ymd
I0910 15:11:18.993607 11401 HdfsTable.java:620] load table raw_data_ymd
I0910 15:11:19.071909 11401 HdfsTable.java:620] load table raw_data_ymd
I0910 15:11:19.147897 11401 frontend.cc:129] com.cloudera.impala.common.AnalysisException: Failed to load metadata for table: partitions.raw_data_ymd
at com.cloudera.impala.analysis.Analyzer.getTable(Analyzer.java:927)
at com.cloudera.impala.analysis.Analyzer.registerBaseTableRef(Analyzer.java:271)
at com.cloudera.impala.analysis.BaseTableRef.analyze(BaseTableRef.java:73)
at com.cloudera.impala.analysis.SelectStmt.analyze(SelectStmt.java:126)
at com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:239)
at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:339)
at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:151)
Caused by: com.cloudera.impala.catalog.TableLoadingException: Failed to load metadata for table: raw_data_ymd
at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:672)
at com.cloudera.impala.catalog.Table.load(Table.java:152)
at com.cloudera.impala.catalog.Db.loadTable(Db.java:80)
at com.cloudera.impala.catalog.Db.access$000(Db.java:44)
at com.cloudera.impala.catalog.Db$1.load(Db.java:56)
at com.cloudera.impala.catalog.Db$1.load(Db.java:52)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
at com.cloudera.impala.catalog.CatalogObjectCache.get(CatalogObjectCache.java:155)
at com.cloudera.impala.catalog.Db.getTable(Db.java:156)
at com.cloudera.impala.catalog.Catalog.getTableInternal(Catalog.java:444)
at com.cloudera.impala.catalog.Catalog.getTable(Catalog.java:465)
at com.cloudera.impala.analysis.Analyzer.getTable(Analyzer.java:913)
... 6 more
Caused by: java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at com.cloudera.impala.catalog.ColumnStats.update(ColumnStats.java:81)
at com.cloudera.impala.catalog.Column.updateStats(Column.java:72)
at com.cloudera.impala.catalog.HdfsTable.loadColumns(HdfsTable.java:396)
at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:642)
... 23 more
I0910 15:11:19.164034 11401 status.cc:44] AnalysisException: Failed to load metadata for table: partitions.raw_data_ymd
CAUSED BY: TableLoadingException: Failed to load metadata for table: raw_data_ymd
CAUSED BY: IllegalStateException: null
@ 0x83af7d (unknown)
@ 0x7059f0 (unknown)
@ 0x6f8866 (unknown)
@ 0x6a1fe1 (unknown)
@ 0x6a2477 (unknown)
@ 0x6e6dfa (unknown)
@ 0x860e10 (unknown)
@ 0x85bc14 (unknown)
@ 0x6aac5e (unknown)
@ 0x126aca9 (unknown)
@ 0x125becf (unknown)
@ 0x125de14 (unknown)
@ 0x1270c52 (unknown)
@ 0x3240a07851 (unknown)
@ 0x32406e894d (unknown)
Suggestion: if the data types of the column stats don't match the current data types of the columns, ignore the stats rather than failing.