[IMPALA-538] Impala stopped to query AVRO tables

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.1
• Fix Version/s: Impala 1.1.1
• Environment: CDH 4.3, CM 4.6.3
• Assignee: Lenni Kuff
• Reporter: Sergey
• Created: 15/Aug/13 09:11
• Updated: 20/Dec/15 00:05
• Resolved: 15/Aug/13 20:31

Description

We've upgraded to Impala 1.1 from 1.0.1
Impala stopped to query avro tables

Example:

CREATE TABLE zone_dimensions
  ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
  WITH SERDEPROPERTIES (
    'avro.schema.url'='hdfs:////applications/avro_schemas/zone_dimensions.avsc')
  STORED as INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'

  LOCATION '/derivative/dict/zone_dimensions'

{

impala-shell:
erver version: impalad version 1.1 RELEASE (build 5e15fcacc48ec4ea65e8aa76362cb3ec9be26f13)
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell v1.1 (5e15fca) built on Sun Jul 21 15:51:04 PDT 2013)

[localhost:21000] > invalidate metadata;
Query: invalidate metadata
Query finished, fetching results ...

[localhost:21000] > refresh zone_dimensions;
Query: refresh zone_dimensions
Query finished, fetching results ...

Returned 0 row(s) in 0.36s
[localhost:21000] > select * from zone_dimensions limit 10;
Query: select * from zone_dimensions limit 10
ERROR: AnalysisException: Failed to load metadata for table: default.zone_dimensions
CAUSED BY: TableLoadingException: Failed to load metadata for table: zone_dimensions
CAUSED BY: NullPointerException: null

impalad logs:

11:30:57.456 INFO com.cloudera.impala.service.Frontend
analyze query select * from zone_dimensions limit 10
11:30:57.485 INFO com.cloudera.impala.catalog.HdfsTable
load table zone_dimensions
11:30:57.716 INFO com.cloudera.impala.catalog.HdfsTable
load block md for zone_dimensions
11:30:57.836 INFO com.cloudera.impala.catalog.HdfsTable
loaded disk ids for table default.zone_dimensions
11:30:57.836 INFO com.cloudera.impala.catalog.HdfsTable
0
11:30:57.836 INFO com.cloudera.impala.catalog.HdfsTable
table #rows=-1
11:30:57.860 INFO com.cloudera.impala.catalog.HdfsTable
load table zone_dimensions
11:30:58.073 INFO com.cloudera.impala.catalog.HdfsTable
load block md for zone_dimensions
11:30:58.176 INFO com.cloudera.impala.catalog.HdfsTable
loaded disk ids for table default.zone_dimensions
11:30:58.176 INFO com.cloudera.impala.catalog.HdfsTable
0
11:30:58.177 INFO com.cloudera.impala.catalog.HdfsTable
table #rows=-1
11:30:58.178 INFO frontend.cc:129
com.cloudera.impala.common.AnalysisException: Failed to load metadata for table: default.zone_dimensions
at com.cloudera.impala.analysis.Analyzer.getTable(Analyzer.java:900)
at com.cloudera.impala.analysis.Analyzer.registerBaseTableRef(Analyzer.java:258)
at com.cloudera.impala.analysis.BaseTableRef.analyze(BaseTableRef.java:73)
at com.cloudera.impala.analysis.SelectStmt.analyze(SelectStmt.java:126)
at com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:233)
at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:339)
at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:133)
Caused by: com.cloudera.impala.catalog.TableLoadingException: Failed to load metadata for table: zone_dimensions
at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:678)
at com.cloudera.impala.catalog.Table.load(Table.java:148)
at com.cloudera.impala.catalog.Db.loadTable(Db.java:79)
at com.cloudera.impala.catalog.Db.access$000(Db.java:43)
at com.cloudera.impala.catalog.Db$1.load(Db.java:55)
at com.cloudera.impala.catalog.Db$1.load(Db.java:51)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
at com.cloudera.impala.catalog.CatalogObjectCache.get(CatalogObjectCache.java:155)
at com.cloudera.impala.catalog.Db.getTable(Db.java:154)
at com.cloudera.impala.catalog.Catalog.getTableInternal(Catalog.java:443)
at com.cloudera.impala.catalog.Catalog.getTable(Catalog.java:464)
at com.cloudera.impala.analysis.Analyzer.getTable(Analyzer.java:893)
... 6 more
Caused by: java.lang.NullPointerException
at com.cloudera.impala.catalog.HdfsTable.getAvroSchema(HdfsTable.java:713)
at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:673)
... 23 more
11:30:58.180 INFO status.cc:44
AnalysisException: Failed to load metadata for table: default.zone_dimensions
CAUSED BY: TableLoadingException: Failed to load metadata for table: zone_dimensions
CAUSED BY: NullPointerException: null
@ 0x83af5d (unknown)
@ 0x702e30 (unknown)
@ 0x6fbe86 (unknown)
@ 0x6ad725 (unknown)
@ 0x6adb97 (unknown)
@ 0x6eb83c (unknown)
@ 0x861110 (unknown)
@ 0x85bf14 (unknown)
@ 0x6b136e (unknown)
@ 0x12aee29 (unknown)
@ 0x129fdcf (unknown)
@ 0x12a1b74 (unknown)
@ 0x12b4dd2 (unknown)
@ 0x33f0407851 (unknown)
@ 0x33f00e811d (unknown)
11:30:58.180 INFO impala-server.cc:951
UnregisterQuery(): query_id=314fd2311d99c028:6e6822ca4419e4b3
11:30:58.180 INFO impala-server.cc:1033
Cancel(): query_id=314fd2311d99c028:6e6822ca4419e4b3