[IMPALA-4798] Invalidate metadata should update missing cache directive info and uncache the respective tables

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.6.0
• Component/s: Catalog
• Labels: ramp-up
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Mala Chikka Kempanna
• Created: 20/Jan/17 20:53
• Updated: 28/Jan/17 01:24

Description

Disabling Hdfs produces following errors in catalogd logs .
This happens because hdfs cache is disabled but corresponding table in impala is not uncached.

An improvement where INVALIDATE METADATA automatically updates tables for missing cache directives by un-caching them would help prevent these errors.

E1121 07:13:23.673859 9107 HdfsCachingUtil.java:490] Cache directive does not exist
Java exception follows:
com.cloudera.impala.common.ImpalaRuntimeException: Did not find requested id 1
at org.apache.hadoop.hdfs.server.namenode.CacheManager.listCacheDirectives(CacheManager.java:704)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.listCacheDirectives(FSNamesystem.java:8833)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.listCacheDirectives(NameNodeRpcServer.java:1551)
at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.listCacheDirectives(AuthorizationProviderProxyClientProtocol.java:804)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.listCacheDirectives(ClientNamenodeProtocolServerSideTranslatorPB.java:1197)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)