[HIVE-22410] CachedStore Prewarm Failure NPE

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Works for Me
• Affects Version/s: 2.3.7, 3.1.2
• Fix Version/s: 3.1.3
• Component/s: Standalone Metastore
• Labels: cachedstore

Environment

JDK 1.7.0_40

• Assignee: Shilun Fan
• Reporter: Shilun Fan
• Created: 27/Oct/19 08:45
• Updated: 04/May/22 11:28
• Resolved: 04/May/22 11:28

Description

java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimateArrayElements(IncrementalObjectSizeEstimator.java:501) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:443) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:433) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:369) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.cached.SharedCache.populateTableInCache(SharedCache.java:1028) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.cached.CachedStore.prewarm(CachedStore.java:314) ~[hive-metastore-2.3.5.jar:2.3.5]
at org.apache.hadoop.hive.metastore.cached.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:492) [hive-metastore-2.3.5.jar:2.3.5]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [?:1.7.0_40]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [?:1.7.0_40]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [?:1.7.0_40]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.7.0_40]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_40]
at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40]

reason:
Methods estimateArrayElements throw NPE

I found that the last class does not appear in the parent (HashMap<Class<?>, Object Estimator> ), and the parent does not contain the information of the ConcurrentHashMap Class.

The solution is to initialize the class information before using it.