[GEODE-1393] locator returns incorrect server information when starting up

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-incubating.M3
• Component/s: locator
• Assignee: Bruce J Schuchardt
• Reporter: Bruce J Schuchardt
• Created: 13/May/16 20:51
• Updated: 22/Aug/16 16:55
• Resolved: 16/May/16 15:17

Description

When starting up a locator has no knowledge of cache servers that might be in the distributed system but it will process server-location requests from clients and return them incorrect information until it receives load info from the servers.

In one test I saw a locator be ejected from the distributed system. When it auto-reconnected some cache clients asked it for server locations and, though there were 6 cache servers available the clients got this exception:

com.gemstone.gemfire.cache.client.NoAvailableServersException
        at com.gemstone.gemfire.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:257)
        at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.getNextOpServerLocation(OpExecutorImpl.java:318)
        at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:130)
        at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:123)
        at com.gemstone.gemfire.cache.client.internal.PoolImpl.execute(PoolImpl.java:714)
        at com.gemstone.gemfire.cache.client.internal.GetOp.execute(GetOp.java:97)
        at com.gemstone.gemfire.cache.client.internal.ServerRegionProxy.get(ServerRegionProxy.java:112)
        at com.gemstone.gemfire.internal.cache.tx.ClientTXRegionStub.findObject(ClientTXRegionStub.java:72)
        at com.gemstone.gemfire.internal.cache.TXStateStub.findObject(TXStateStub.java:379)
        at com.gemstone.gemfire.internal.cache.TXStateProxyImpl.findObject(TXStateProxyImpl.java:607)
        at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1460)
        at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1398)
        at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1385)
        at com.gemstone.gemfire.internal.cache.AbstractRegion.get(AbstractRegion.java:336)

ServerLocator has a readiness check but it is only testing to see if its DistributedSystem instance variable has been initialized. It ought to wait until it has received a server load update.