[IMPALA-5540] Latest version of Sentry fails to connect

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 2.10.0
• Component/s: Infrastructure
• Epic Color: ghx-label-3
• Assignee: Thomas Tauber-Marshall
• Reporter: Thomas Tauber-Marshall
• Created: 20/Jun/17 13:51
• Updated: 21/Jun/17 21:40
• Resolved: 21/Jun/17 21:40

Description

As far as I can tell, Sentry does start up correctly. Output of bin/sentry:

log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout.
371 T1 oasst.SentryService.<init> Configured on address localhost/127.0.0.1:30911
601 T1 odu.Log4JLogger.info Property datanucleus.cache.level2 unknown - will be ignored
827 T1 cjb.BoneCPConfig.sanitize WARN Max Connections < 1. Setting to 20
2299 T1 cjb.BoneCPConfig.sanitize WARN Max Connections < 1. Setting to 20
2336 T1 oasst.LeaderStatusMonitor.<init> Leader election protocol disabled, assuming single active server
2337 T1 oasst.SentryService.start Attempting to start...
2339 T26 oaspdsp.SentryStore.purgeDeltaChangeTables Purging MSentryPathUpdate and MSentyPermUpdate tables, leaving 200 entries
2339 T24 oasst.SentryService.startSentryStoreCleaner sentry store cleaner is scheduled with interval 43200 seconds
2339 T24 oasst.SentryService.startHMSFollower HMS follower is not started because HDFS sync is disabled.
2349 T24 oasst.SentryService.runServer ProcessorFactory being used: org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessorFactory
2362 T24 oasst.SentryService.runServer ProcessorFactory being used: org.apache.sentry.provider.db.generic.service.thrift.SentryGenericPolicyProcessorFactory
2417 T24 odu.Log4JLogger.info Property datanucleus.cache.level2 unknown - will be ignored
2483 T26 oaspdsp.SentryStore.purgeDeltaChangeTableCore Purged 0 of MSentryPermChange to changeID=-200
2484 T26 oaspdsp.SentryStore$9.execute MSentryPermChange table has been purged.
2608 T26 oaspdsp.SentryStore.purgeDeltaChangeTableCore Purged 0 of MSentryPathChange to changeID=-200
2608 T26 oaspdsp.SentryStore$9.execute MSentryPathUpdate table has been purged.
2905 T24 cjb.BoneCPConfig.sanitize WARN Max Connections < 1. Setting to 20
3011 T24 cjb.BoneCPConfig.sanitize WARN Max Connections < 1. Setting to 20
3038 T24 oasst.SentryService.runServer Serving on localhost/127.0.0.1:30911
3038 T24 oasst.SentryService.runServer Sentry service is ready to serve client requests

but we fail to connect to it. output of SentryServicePinger:

Stopping Sentry
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/dev/Impala/fe/target/dependency/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/dev/Impala/testdata/target/dependency/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/06/20 06:42:14 INFO transport.SentryTransportPool: Creating pool for localhost with default port 30911
17/06/20 06:42:14 INFO transport.SentryTransportPool: Adding endpoint localhost:30911
17/06/20 06:42:14 INFO transport.SentryTransportPool: Connection pooling is enabled
17/06/20 06:42:14 ERROR transport.SentryTransportPool: Failed to obtain transport for localhost:30911: java.net.ConnectException: Connection refused
17/06/20 06:42:14 ERROR transport.RetryClientInvocationHandler: Failed to connect
org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
        at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
        at org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99)
        at org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86)
        at org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:282)
        at org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:251)
        at org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
        at org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
        at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380)
        at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279)
        at org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:163)
        at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:85)
        at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:139)
        at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:88)
        at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
        at com.sun.proxy.$Proxy0.listRoles(Unknown Source)
        at org.apache.impala.util.SentryPolicyService.listAllRoles(SentryPolicyService.java:393)
        at org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:75)
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
        ... 16 more