[ATLAS-4549] Issue with brining up Atlas in HA enabled mode with Index recovery enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0, 2.2.0
• Fix Version/s: 3.0.0, 2.2.0
• Assignee: Radhika Kundam
• Reporter: Radhika Kundam
• Created: 04/Feb/22 21:28
• Updated: 08/Mar/22 19:36
• Resolved: 08/Mar/22 19:36

Description

With HA enabled and Index Recovery enabled, Atlas migration is having issues.

2022-02-03 02:32:11,790 INFO  - [main-EventThread:] ~ ==> IndexRecoveryService.instanceIsActive() (IndexRecoveryService:117)
2022-02-03 02:32:11,791 ERROR - [main-EventThread:] ~ Got exception while activating (ActiveInstanceElectorService:162)
java.lang.IllegalThreadStateException
    at java.lang.Thread.start(Thread.java:710)
    at org.apache.atlas.repository.graph.IndexRecoveryService.startTxLogMonitoring(IndexRecoveryService.java:141)
    at org.apache.atlas.repository.graph.IndexRecoveryService.instanceIsActive(IndexRecoveryService.java:119)
    at org.apache.atlas.web.service.ActiveInstanceElectorService.isLeader(ActiveInstanceElectorService.java:156)
    at org.apache.curator.framework.recipes.leader.LeaderLatch$9.apply(LeaderLatch.java:702)
    at org.apache.curator.framework.recipes.leader.LeaderLatch$9.apply(LeaderLatch.java:698)
    at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:100)
    at org.apache.curator.shaded.com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
    at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:92)
    at org.apache.curator.framework.recipes.leader.LeaderLatch.setLeadership(LeaderLatch.java:697)
    at org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:575)
    at org.apache.curator.framework.recipes.leader.LeaderLatch.access$600(LeaderLatch.java:65)
    at org.apache.curator.framework.recipes.leader.LeaderLatch$7.processResult(LeaderLatch.java:626)
    at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:910)
    at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:653)
    at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152)
    at org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:187)
    at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:624)
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:507)
2022-02-03 02:32:11,791 WARN  - [main-EventThread:] ~ Server instance with server id id2 is removed as leader (ActiveInstanceElectorService:199)
2022-02-03 02:32:11,791 WARN  - [main-EventThread:] ~ Instance becoming passive from BECOMING_ACTIVE (ServiceState:119)
2022-02-03 02:32:11,791 INFO  - [main-EventThread:] ~ IndexRecoveryService.instanceIsPassive(): no action needed. (IndexRecoveryService:126)

Issue Links

• links to: Code Review