[IGNITE-15284] NPE when exporting metrics on deactivation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.10, 2.9.1
• Fix Version/s: 2.11
• Labels: IEP-35 ise
• Assignee: Vladimir Steshin
• Reporter: Nikolay Izhikov
• Created: 11/Aug/21 07:03
• Updated: 16/Sep/21 15:21
• Resolved: 16/Sep/21 15:20

Description

Some production installation has the following trace in logs

[14:38:53,321][INFO][disco-notifier-worker-#44][GridClusterStateProcessor] Received state change finish message: INACTIVE
[14:38:53,321][INFO][disco-notifier-worker-#44][GridClusterStateProcessor] Cluster state was changed from ACTIVE to INACTIVE
[14:38:53,322][INFO][sys-#2263][GridCacheProcessor] Stopped cache [cacheName=ignite-sys-cache]
[14:38:53,323][INFO][db-checkpoint-thread-#760][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointBeforeLockTime=0ms, checkpointLockWait=0ms, checkpointListenersExecuteTime=0ms, checkpointLockHoldTime=0ms, reason='timeout']
[14:38:53,361][INFO][sys-#2263][FilePageStoreManager] Cleanup cache stores [total=2, left=0, cleanFiles=false]
[14:38:53,361][INFO][exchange-worker-#51][GridCachePartitionExchangeManager] Skipping rebalancing (no affinity changes) [top=AffinityTopologyVersion [topVer=30, minorTopVer=9], evt=DISCOVERY_CUSTOM_EVT, evtNode=b6e5f6cf-2fe9-4568-957f-e03d8e00b170, client=false]
[14:38:53,361][INFO][sys-#2263][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=db5f603b-6ce6-4efc-9a9d-bffec75af216, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=30, minorTopVer=9], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=b6e5f6cf-2fe9-4568-957f-e03d8e00b170, consistentId=7cb5661c-e1f9-4d90-987c-4098b10599f1, addrs=ArrayList [10.3.106.165, 127.0.0.1], sockAddrs=HashSet [/10.3.106.165:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1628576506706, loc=false, ver=2.9.1#20201203-sha1:adcce517, isClient=false], rebalanced=false, done=true, newCrdFut=null], topVer=AffinityTopologyVersion [topVer=30, minorTopVer=9]]
[14:38:53,361][INFO][sys-#2263][GridDhtPartitionsExchangeFuture] Exchange timings [startVer=AffinityTopologyVersion [topVer=30, minorTopVer=9], resVer=AffinityTopologyVersion [topVer=30, minorTopVer=9], stage="Waiting in exchange queue" (0 ms), stage="Exchange parameters initialization" (0 ms), stage="Update caches registry" (0 ms), stage="Start caches" (0 ms), stage="Affinity initialization on cache group start" (0 ms), stage="Determine exchange type" (0 ms), stage="Preloading notification" (0 ms), stage="Wait partitions release [latch=exchange]" (0 ms), stage="Wait partitions release latch [latch=exchange]" (11 ms), stage="Wait partitions release [latch=exchange]" (0 ms), stage="After states restored callback" (0 ms), stage="WAL history reservation" (0 ms), stage="Waiting for Full message" (3 ms), stage="Affinity recalculation" (0 ms), stage="Full map updating" (0 ms), stage="Exchange done" (41 ms), stage="Total time" (55 ms)]
[14:38:53,361][INFO][sys-#2263][GridDhtPartitionsExchangeFuture] Exchange longest local stages [startVer=AffinityTopologyVersion [topVer=30, minorTopVer=9], resVer=AffinityTopologyVersion [topVer=30, minorTopVer=9]]
[14:38:54,134][WARNING][pool-9-thread-1][GridMetricManager] Exception in supplier
java.lang.NullPointerException
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.initForCache(FilePageStoreManager.java:673)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.lambda$getHolder$7(FilePageStoreManager.java:1260)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager$IdxCacheStores.lambda$computeIfAbsent$668d01b6$1(FilePageStoreManager.java:1535)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager$LongOperationAsyncExecutor.afterAsyncCompletion(FilePageStoreManager.java:1461)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager$IdxCacheStores.computeIfAbsent(FilePageStoreManager.java:1535)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.getHolder(FilePageStoreManager.java:1253)
	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.getStores(FilePageStoreManager.java:1283)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.forGroupPageStores(GridCacheDatabaseSharedManager.java:2135)
	at org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl.lambda$new$0(CacheGroupMetricsImpl.java:102)
	at org.apache.ignite.internal.util.lang.GridFunc.lambda$nonThrowableSupplier$3(GridFunc.java:3341)
	at org.apache.ignite.internal.processors.metric.impl.LongGauge.value(LongGauge.java:45)
	at org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi.lambda$null$5(OpenCensusMetricExporterSpi.java:161)
	at java.base/java.lang.Iterable.forEach(Unknown Source)
	at org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi.lambda$export$6(OpenCensusMetricExporterSpi.java:152)

Issue Links

• is fixed by: IGNITE-14990 Incorrect values of cache, cache group, data region metrics after cluster re-activation