[2017-04-03 10:01:37,000] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [2017-04-03 10:01:37,003] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager) [2017-04-03 10:01:37,003] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager) [2017-04-03 10:01:37,003] INFO Purge task is not scheduled. (org.apache.zookeeper.server.DatadirCleanupManager) [2017-04-03 10:01:37,003] WARN Either no config or no quorum defined in config, running in standalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain) [2017-04-03 10:01:37,021] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [2017-04-03 10:01:37,021] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain) [2017-04-03 10:01:37,029] INFO Server environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:host.name=kafka.example.com (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.version=1.8.0_121 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.vendor=Oracle Corporation (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.home=/usr/lib/jvm/java-8-oracle/jre (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.class.path=:/etc/kafka/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/etc/kafka/bin/../libs/argparse4j-0.5.0.jar:/etc/kafka/bin/../libs/connect-api-0.10.1.1.jar:/etc/kafka/bin/../libs/connect-file-0.10.1.1.jar:/etc/kafka/bin/../libs/connect-json-0.10.1.1.jar:/etc/kafka/bin/../libs/connect-runtime-0.10.1.1.jar:/etc/kafka/bin/../libs/guava-18.0.jar:/etc/kafka/bin/../libs/hk2-api-2.4.0-b34.jar:/etc/kafka/bin/../libs/hk2-locator-2.4.0-b34.jar:/etc/kafka/bin/../libs/hk2-utils-2.4.0-b34.jar:/etc/kafka/bin/../libs/jackson-annotations-2.6.0.jar:/etc/kafka/bin/../libs/jackson-core-2.6.3.jar:/etc/kafka/bin/../libs/jackson-databind-2.6.3.jar:/etc/kafka/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/etc/kafka/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/etc/kafka/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/etc/kafka/bin/../libs/javassist-3.18.2-GA.jar:/etc/kafka/bin/../libs/javax.annotation-api-1.2.jar:/etc/kafka/bin/../libs/javax.inject-1.jar:/etc/kafka/bin/../libs/javax.inject-2.4.0-b34.jar:/etc/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/etc/kafka/bin/../libs/javax.ws.rs-api-2.0.1.jar:/etc/kafka/bin/../libs/jersey-client-2.22.2.jar:/etc/kafka/bin/../libs/jersey-common-2.22.2.jar:/etc/kafka/bin/../libs/jersey-container-servlet-2.22.2.jar:/etc/kafka/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/etc/kafka/bin/../libs/jersey-guava-2.22.2.jar:/etc/kafka/bin/../libs/jersey-media-jaxb-2.22.2.jar:/etc/kafka/bin/../libs/jersey-server-2.22.2.jar:/etc/kafka/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-http-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-io-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-security-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-server-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jetty-util-9.2.15.v20160210.jar:/etc/kafka/bin/../libs/jopt-simple-4.9.jar:/etc/kafka/bin/../libs/kafka_2.11-0.10.1.1.jar:/etc/kafka/bin/../libs/kafka_2.11-0.10.1.1-sources.jar:/etc/kafka/bin/../libs/kafka_2.11-0.10.1.1-test-sources.jar:/etc/kafka/bin/../libs/kafka-clients-0.10.1.1.jar:/etc/kafka/bin/../libs/kafka-log4j-appender-0.10.1.1.jar:/etc/kafka/bin/../libs/kafka-streams-0.10.1.1.jar:/etc/kafka/bin/../libs/kafka-streams-examples-0.10.1.1.jar:/etc/kafka/bin/../libs/kafka-tools-0.10.1.1.jar:/etc/kafka/bin/../libs/log4j-1.2.17.jar:/etc/kafka/bin/../libs/lz4-1.3.0.jar:/etc/kafka/bin/../libs/metrics-core-2.2.0.jar:/etc/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/etc/kafka/bin/../libs/reflections-0.9.10.jar:/etc/kafka/bin/../libs/rocksdbjni-4.9.0.jar:/etc/kafka/bin/../libs/scala-library-2.11.8.jar:/etc/kafka/bin/../libs/scala-parser-combinators_2.11-1.0.4.jar:/etc/kafka/bin/../libs/slf4j-api-1.7.21.jar:/etc/kafka/bin/../libs/slf4j-log4j12-1.7.21.jar:/etc/kafka/bin/../libs/snappy-java-1.1.2.6.jar:/etc/kafka/bin/../libs/validation-api-1.1.0.Final.jar:/etc/kafka/bin/../libs/zkclient-0.9.jar:/etc/kafka/bin/../libs/zookeeper-3.4.8.jar (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:java.compiler= (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:os.version=4.4.0-31-generic (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,029] INFO Server environment:user.name=sliman (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,030] INFO Server environment:user.home=/home/sliman (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,030] INFO Server environment:user.dir=/etc/kafka (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,037] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,037] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:01:37,037] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer) >>> KeyTabInputStream, readName(): EXAMPLE.COM >>> KeyTabInputStream, readName(): zookeeper >>> KeyTabInputStream, readName(): kafka.example.com >>> KeyTab: load() entry length: 82; type: 16 Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Java config name: /etc/krb5.conf Loaded from Java config Added key: 16version: 3 >>> KdcAccessibility: reset Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 default etypes for default_tkt_enctypes: 16. >>> KrbAsReq creating message >>> KrbKdcReq send: kdc=kafka.example.com UDP:88, timeout=30000, number of retries =3, #bytes=157 >>> KDCCommunication: kdc=kafka.example.com UDP:88, timeout=30000,Attempt =1, #bytes=157 >>> KrbKdcReq send: #bytes read=362 >>>Pre-Authentication Data: PA-DATA type = 136 >>>Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 16, salt = EXAMPLE.COMzookeeperkafka.example.com >>>Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 16, salt = EXAMPLE.COMzookeeperkafka.example.com, s2kparams = null >>>Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP >>>Pre-Authentication Data: PA-DATA type = 133 >>> KdcAccessibility: remove kafka.example.com:88 >>> KDCRep: init() encoding tag is 126 req type is 11 >>>KRBError: cTime is Thu Jul 07 22:40:31 WEST 2011 1310074831000 sTime is Mon Apr 03 10:01:37 WEST 2017 1491210097000 suSec is 131042 error code is 25 error Message is Additional pre-authentication required cname is zookeeper/kafka.example.com@EXAMPLE.COM sname is krbtgt/EXAMPLE.COM@EXAMPLE.COM eData provided. msgType is 30 >>>Pre-Authentication Data: PA-DATA type = 136 >>>Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 16, salt = EXAMPLE.COMzookeeperkafka.example.com >>>Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 16, salt = EXAMPLE.COMzookeeperkafka.example.com, s2kparams = null >>>Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP >>>Pre-Authentication Data: PA-DATA type = 133 KRBError received: NEEDED_PREAUTH KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ default etypes for default_tkt_enctypes: 16. Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 default etypes for default_tkt_enctypes: 16. >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType >>> KrbAsReq creating message >>> KrbKdcReq send: kdc=kafka.example.com UDP:88, timeout=30000, number of retries =3, #bytes=248 >>> KDCCommunication: kdc=kafka.example.com UDP:88, timeout=30000,Attempt =1, #bytes=248 >>> KrbKdcReq send: #bytes read=833 >>> KdcAccessibility: remove kafka.example.com:88 Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType >>> KrbAsRep cons in KrbAsReq.getReply zookeeper/kafka.example.com [2017-04-03 10:01:37,280] INFO successfully logged in. (org.apache.zookeeper.Login) [2017-04-03 10:01:37,287] INFO binding to port 0.0.0.0/0.0.0.0:2182 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2017-04-03 10:01:37,288] INFO TGT refresh thread started. (org.apache.zookeeper.Login) [2017-04-03 10:01:37,296] INFO TGT valid starting at: Mon Apr 03 10:01:37 WEST 2017 (org.apache.zookeeper.Login) [2017-04-03 10:01:37,296] INFO TGT expires: Mon Apr 03 20:01:37 WEST 2017 (org.apache.zookeeper.Login) [2017-04-03 10:01:37,296] INFO TGT refresh sleeping until: Mon Apr 03 18:11:40 WEST 2017 (org.apache.zookeeper.Login) [2017-04-03 10:03:41,416] INFO Accepted socket connection from /127.0.0.1:47000 (org.apache.zookeeper.server.NIOServerCnxnFactory) Found KeyTab /etc/kafka/keytabs/zookeeper.keytab for zookeeper/kafka.example.com@EXAMPLE.COM Found ticket for zookeeper/kafka.example.com@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Mon Apr 03 20:01:37 WEST 2017 [2017-04-03 10:03:41,441] INFO Client attempting to establish new session at /127.0.0.1:47000 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:03:41,443] INFO Creating new log file: log.2a (org.apache.zookeeper.server.persistence.FileTxnLog) [2017-04-03 10:03:41,451] INFO Established session 0x15b330c52b30000 with negotiated timeout 6000 for client /127.0.0.1:47000 (org.apache.zookeeper.server.ZooKeeperServer) Entered Krb5Context.acceptSecContext with state=STATE_NEW Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType default etypes for permitted_enctypes: 16. >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType MemoryCache: add 1491210221/474202/9ECA4C4745FF2D5C2D9A03FC4FC5564A/kafka/kafka.example.com@EXAMPLE.COM to kafka/kafka.example.com@EXAMPLE.COM|zookeeper/kafka.example.com@EXAMPLE.COM >>> KrbApReq: authenticate succeed. Krb5Context setting peerSeqNumber to: 413519251 Krb5Context setting mySeqNumber to: 413519251 Krb5Context.wrap: data=[01 01 00 00 ] Krb5Context.wrap: token=[60 3f 06 09 2a 86 48 86 f7 12 01 02 02 02 01 04 00 ff ff ff ff 61 c2 c2 3f c4 3f cf c7 48 9d cd b2 78 af c6 1c 46 c8 54 53 55 9e 67 44 9c 8a be 6a cd 17 da 12 0e 9d 05 1f 01 01 00 00 04 04 04 04 ] Krb5Context.unwrap: token=[60 5f 06 09 2a 86 48 86 f7 12 01 02 02 02 01 04 00 ff ff ff ff 1b 08 ed 4a 75 f6 bf 75 89 b4 04 f9 05 ba fe fa c2 bd da 03 d8 ca 58 28 27 8c 46 03 d6 52 54 c8 0d ab 49 29 01 01 00 00 6b 61 66 6b 61 2f 6b 61 66 6b 61 2e 65 78 61 6d 70 6c 65 2e 63 6f 6d 40 45 58 41 4d 50 4c 45 2e 43 4f 4d 01 ] Krb5Context.unwrap: data=[01 01 00 00 6b 61 66 6b 61 2f 6b 61 66 6b 61 2e 65 78 61 6d 70 6c 65 2e 63 6f 6d 40 45 58 41 4d 50 4c 45 2e 43 4f 4d ] [2017-04-03 10:03:41,508] INFO Successfully authenticated client: authenticationID=kafka/kafka.example.com@EXAMPLE.COM; authorizationID=kafka/kafka.example.com@EXAMPLE.COM. (org.apache.zookeeper.server.auth.SaslServerCallbackHandler) [2017-04-03 10:03:41,534] INFO Setting authorizedID: kafka/kafka.example.com@EXAMPLE.COM (org.apache.zookeeper.server.auth.SaslServerCallbackHandler) [2017-04-03 10:03:41,534] INFO adding SASL authorization for authorizationID: kafka/kafka.example.com@EXAMPLE.COM (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:03:42,120] INFO Got user-level KeeperException when processing sessionid:0x15b330c52b30000 type:delete cxid:0x23 zxid:0x2d txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election (org.apache.zookeeper.server.PrepRequestProcessor) [2017-04-03 10:03:42,164] INFO Accepted socket connection from /127.0.0.1:47002 (org.apache.zookeeper.server.NIOServerCnxnFactory) Found KeyTab /etc/kafka/keytabs/zookeeper.keytab for zookeeper/kafka.example.com@EXAMPLE.COM Found ticket for zookeeper/kafka.example.com@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Mon Apr 03 20:01:37 WEST 2017 [2017-04-03 10:03:42,166] INFO Client attempting to establish new session at /127.0.0.1:47002 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:03:42,174] INFO Established session 0x15b330c52b30001 with negotiated timeout 6000 for client /127.0.0.1:47002 (org.apache.zookeeper.server.ZooKeeperServer) Entered Krb5Context.acceptSecContext with state=STATE_NEW Looking for keys for: zookeeper/kafka.example.com@EXAMPLE.COM Added key: 16version: 3 >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType default etypes for permitted_enctypes: 16. >>> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType MemoryCache: add 1491210222/175990/9EDD5607BF9D2E19F91C90477FBB6C90/kafka/kafka.example.com@EXAMPLE.COM to kafka/kafka.example.com@EXAMPLE.COM|zookeeper/kafka.example.com@EXAMPLE.COM MemoryCache: Existing AuthList: #1: 1491210221/474202/9ECA4C4745FF2D5C2D9A03FC4FC5564A/kafka/kafka.example.com@EXAMPLE.COM >>> KrbApReq: authenticate succeed. Krb5Context setting peerSeqNumber to: 1066216531 Krb5Context setting mySeqNumber to: 1066216531 Krb5Context.wrap: data=[01 01 00 00 ] Krb5Context.wrap: token=[60 3f 06 09 2a 86 48 86 f7 12 01 02 02 02 01 04 00 ff ff ff ff af 94 da d5 70 37 fe 96 fe 44 28 b6 3f e6 4a f5 b1 a4 94 2f b8 27 da a9 51 79 1b ad 82 85 81 b9 bb f7 89 39 01 01 00 00 04 04 04 04 ] Krb5Context.unwrap: token=[60 5f 06 09 2a 86 48 86 f7 12 01 02 02 02 01 04 00 ff ff ff ff d9 b5 e6 9d 93 e7 f6 01 df 0f c1 dc 1a 58 03 c4 48 ee 57 b7 b4 13 7e 48 bd 97 e6 0f 37 71 1f 56 78 a0 65 36 01 01 00 00 6b 61 66 6b 61 2f 6b 61 66 6b 61 2e 65 78 61 6d 70 6c 65 2e 63 6f 6d 40 45 58 41 4d 50 4c 45 2e 43 4f 4d 01 ] Krb5Context.unwrap: data=[01 01 00 00 6b 61 66 6b 61 2f 6b 61 66 6b 61 2e 65 78 61 6d 70 6c 65 2e 63 6f 6d 40 45 58 41 4d 50 4c 45 2e 43 4f 4d ] [2017-04-03 10:03:42,188] INFO Successfully authenticated client: authenticationID=kafka/kafka.example.com@EXAMPLE.COM; authorizationID=kafka/kafka.example.com@EXAMPLE.COM. (org.apache.zookeeper.server.auth.SaslServerCallbackHandler) [2017-04-03 10:03:42,188] INFO Setting authorizedID: kafka/kafka.example.com@EXAMPLE.COM (org.apache.zookeeper.server.auth.SaslServerCallbackHandler) [2017-04-03 10:03:42,188] INFO adding SASL authorization for authorizationID: kafka/kafka.example.com@EXAMPLE.COM (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-03 10:03:42,250] INFO Got user-level KeeperException when processing sessionid:0x15b330c52b30000 type:create cxid:0x2e zxid:0x2f txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor) [2017-04-03 10:03:42,251] INFO Got user-level KeeperException when processing sessionid:0x15b330c52b30000 type:create cxid:0x2f zxid:0x30 txntype:-1 reqpath:n/a Error Path:/brokers/ids Error:KeeperErrorCode = NodeExists for /brokers/ids (org.apache.zookeeper.server.PrepRequestProcessor)