[HIVE-21479] NPE during metastore cache update

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Hadoop Flags: Reviewed
• Assignee: Daniel Dai Assign to me
• Reporter: Daniel Dai
• Created: 19/Mar/19 17:35
• Updated: 17/Nov/22 08:49
• Resolved: 26/Mar/19 21:34

Description

Saw the following stack during a long periodical update:

2019-03-12T10:01:43,015 ERROR [CachedStore-CacheUpdateService: Thread-36] cache.CachedStore: Update failure:java.lang.NullPointerException
	at org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.updateTableColStats(CachedStore.java:508)
	at org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.update(CachedStore.java:461)
	at org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:396)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

The reason is we get the table list at very early stage and then refresh table one by one. It is likely table is removed during the interim. We need to deal with this case during cache update.