Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-715

NumberFormatException in PartitionStateInfo

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.8.0
    • None
    • replication
    • None

    Description

      Hey Guys,

      During a broker restart, I got this exception:

      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:host.name=eat1-qa466.corp.linkedin.com
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.version=1.6.0_21
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.vendor=Sun Microsystems Inc.
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.home=/export/apps/jdk/JDK-1_6_0_21/jre
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.class.path=/export/apps/jdk/JDK-1_6_0_21/lib/tools.jar:lib/activation-1.0.2.jar:lib/ant-1.6.5.jar:lib/aopalliance-1.0.jar:lib/cfg-2.8.0.jar:lib/cfg-api-6.6.6.jar:lib/cfg-impl-6.6.6.jar:lib/com.linkedin.customlibrary.j2ee-1.0.jar:lib/com.linkedin.customlibrary.mx4j-3.0.2.jar:lib/com.linkedin.customlibrary.xmsg-0.6.jar:lib/commons-beanutils-1.7.0.jar:lib/commons-cli-1.0.jar:lib/commons-lang-2.4.jar:lib/commons-logging-1.1.jar:lib/configuration-api-1.4.8.jar:lib/configuration-repository-impl-1.4.8.jar:lib/container-management-impl-1.1.15.jar:lib/container-server-1.1.15.jar:lib/emweb-impl-1.1.15.jar:lib/jaxen-1.1.1.jar:lib/jdom-1.0.jar:lib/jetty-6.1.26.jar:lib/jetty-management-6.1.26.jar:lib/jetty-naming-6.1.26.jar:lib/jetty-plus-6.1.26.jar:lib/jetty-util5-6.1.26.jar:lib/jetty-util-6.1.26.jar:lib/jmx-impl-1.4.8.jar:lib/json-simple-1.1.jar:lib/jsp-2.1-6.1.1.jar:lib/jsp-api-2.1-6.1.1.jar:lib/lispring-lispring-core-1.4.8.jar:lib/lispring-lispring-servlet-1.4.8.jar:lib/log4j-1.2.15.jar:lib/mail-1.3.0.jar:lib/mx4j-tools-3.0.2.jar:lib/servlet-api-2.5.jar:lib/spring-aop-3.0.3.jar:lib/spring-asm-3.0.3.jar:lib/spring-aspects-3.0.3.jar:lib/spring-beans-3.0.3.jar:lib/spring-context-3.0.3.jar:lib/spring-context-support-3.0.3.jar:lib/spring-core-3.0.3.jar:lib/spring-expression-3.0.3.jar:lib/spring-jdbc-3.0.3.jar:lib/spring-jms-3.0.3.jar:lib/spring-orm-3.0.3.jar:lib/spring-transaction-3.0.3.jar:lib/spring-web-3.0.3.jar:lib/spring-web-servlet-3.0.3.jar:lib/util-core-4.0.40.jar:lib/util-i18n-4.0.40.jar:lib/util-jmx-4.0.22.jar:lib/util-log-4.0.40.jar:lib/util-servlet-4.0.40.jar:lib/util-xmsg-4.0.40.jar:lib/xml-apis-1.3.04.jar
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.library.path=/export/apps/jdk/JDK-1_6_0_21/jre/lib/amd64/server:/export/apps/jdk/JDK-1_6_0_21/jre/lib/amd64:/export/apps/jdk/JDK-1_6_0_21/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.io.tmpdir=/export/content/glu/apps/kafka/i001/tmp
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:java.compiler=<NA>
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:os.name=Linux
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:os.arch=amd64
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:os.version=2.6.32-220.13.1.el6.x86_64
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:user.name=app
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:user.home=/home/app
      2013/01/21 19:21:10.918 INFO [ZooKeeper] [main] [kafka] [] Client environment:user.dir=/export/content/glu/apps/kafka/i001
      2013/01/21 19:21:10.919 INFO [ZooKeeper] [main] [kafka] [] Initiating client connection, connectString=eat1-app309.corp.linkedin.com:12913,eat1-app310.corp.linkedin.com:12913,eat1-app311.corp.linkedin.com:12913,eat1-app312.corp.linkedin.com:12913,eat1-app313.corp.linkedin.com:12913/kafka-samsa sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1bfdbab5
      2013/01/21 19:21:10.932 INFO [ClientCnxn] [main-SendThread()] [kafka] [] Opening socket connection to server eat1-app313.corp.linkedin.com/172.20.72.73:12913
      2013/01/21 19:21:10.933 INFO [ClientCnxn] [main-SendThread(eat1-app313.corp.linkedin.com:12913)] [kafka] [] Socket connection established to eat1-app313.corp.linkedin.com/172.20.72.73:12913, initiating session
      2013/01/21 19:21:10.963 INFO [ClientCnxn] [main-SendThread(eat1-app313.corp.linkedin.com:12913)] [kafka] [] Session establishment complete on server eat1-app313.corp.linkedin.com/172.20.72.73:12913, sessionid = 0x53afd073784059c, negotiated timeout = 6000
      2013/01/21 19:21:10.964 INFO [ZkClient] [main-EventThread] [kafka] [] zookeeper state changed (SyncConnected)
      2013/01/21 19:21:10.979 INFO [ZkUtils$] [main] [kafka] [] Registered broker 466 at path /brokers/ids/466 with address eat1-qa466.corp.linkedin.com:10251.
      2013/01/21 19:21:10.979 INFO [KafkaServer] [main] [kafka] [] [Kafka Server 466], Connecting to ZK: eat1-app309.corp.linkedin.com:12913,eat1-app310.corp.linkedin.com:12913,eat1-app311.corp.linkedin.com:12913,eat1-app312.corp.linkedin.com:12913,eat1-app313.corp.linkedin.com:12913/kafka-samsa
      2013/01/21 19:21:11.018 INFO [ControllerEpochListener] [main] [kafka] [] [ControllerEpochListener on 466]: Initialized controller epoch to 22 and zk version 21
      2013/01/21 19:21:11.054 ERROR [Processor] [kafka-processor-10251-0] [kafka] [] Closing socket for /172.18.146.131 because of error
      java.lang.NumberFormatException: For input string: ""
      at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
      at java.lang.Integer.parseInt(Integer.java:470)
      at java.lang.Integer.parseInt(Integer.java:499)
      at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:207)
      at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
      at kafka.api.PartitionStateInfo$$anonfun$1.apply(LeaderAndIsrRequest.scala:51)
      at kafka.api.PartitionStateInfo$$anonfun$1.apply(LeaderAndIsrRequest.scala:51)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
      at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
      at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
      at scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34)
      at kafka.api.PartitionStateInfo$.readFrom(LeaderAndIsrRequest.scala:51)
      at kafka.api.LeaderAndIsrRequest$$anonfun$readFrom$2.apply(LeaderAndIsrRequest.scala:100)
      at kafka.api.LeaderAndIsrRequest$$anonfun$readFrom$2.apply(LeaderAndIsrRequest.scala:97)
      at scala.collection.immutable.Range$ByOne$class.foreach(Range.scala:282)
      at scala.collection.immutable.Range$$anon$2.foreach(Range.scala:265)
      at kafka.api.LeaderAndIsrRequest$.readFrom(LeaderAndIsrRequest.scala:97)
      at kafka.api.RequestKeys$$anonfun$5.apply(RequestKeys.scala:36)
      at kafka.api.RequestKeys$$anonfun$5.apply(RequestKeys.scala:36)
      at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:47)
      at kafka.network.Processor.read(SocketServer.scala:320)
      at kafka.network.Processor.run(SocketServer.scala:231)
      at java.lang.Thread.run(Thread.java:619)
      2013/01/21 19:21:11.071 INFO [Mx4jLoader$] [main] [kafka] [] mx4j successfuly loaded
      2013/01/21 19:21:11.073 INFO [KafkaController] [main] [kafka] [] [Controller 466]: Controller starting up
      2013/01/21 19:21:11.091 INFO [ZkUtils$] [main] [kafka] [] conflict in /controller data: 466 stored data: 465
      2013/01/21 19:21:11.093 INFO [KafkaController] [main] [kafka] [] [Controller 466]: Controller startup complete
      2013/01/21 19:21:11.099 INFO [KafkaServer] [main] [kafka] [] [Kafka Server 466], started
      2013/01/21 19:21:11.099 INFO [VerifiableProperties] [main] [kafka] [] Verifying properties
      2013/01/21 19:21:11.100 WARN [VerifiableProperties] [main] [kafka] [] Property zk.sessiontimeout.ms is not valid
      2013/01/21 19:21:11.100 INFO [VerifiableProperties] [main] [kafka] [] Property num.replica.fetchers is overridden to 1
      2013/01/21 19:21:11.100 INFO [VerifiableProperties] [main] [kafka] [] Property log.retention.hours is overridden to 168
      2013/01/21 19:21:11.100 WARN [VerifiableProperties] [main] [kafka] [] Property zk.connectiontimeout.ms is not valid
      2013/01/21 19:21:11.100 INFO [VerifiableProperties] [main] [kafka] [] Property auto.create.topics.enable is overridden to true
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property replica.fetch.min.bytes is overridden to 1
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property replica.lag.time.max.ms is overridden to 10000
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property log.retention.bytes is overridden to -1
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property log.flush.interval.messages is overridden to 10000
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property socket.request.max.bytes is overridden to 104857600
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property default.replication.factor is overridden to 3
      2013/01/21 19:21:11.101 INFO [VerifiableProperties] [main] [kafka] [] Property replica.fetch.wait.max.ms is overridden to 500
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property log.cleanup.interval.mins is overridden to 30
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property num.partitions is overridden to 12
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property log.segment.bytes.per.topic is overridden to
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property controller.socket.timeout.ms is overridden to 30000
      2013/01/21 19:21:11.102 WARN [VerifiableProperties] [main] [kafka] [] Property log.flush.intervals.ms.per.topic is not valid
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property socket.receive.buffer.bytes is overridden to 1048576
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property queued.max.requests is overridden to 16
      2013/01/21 19:21:11.102 INFO [VerifiableProperties] [main] [kafka] [] Property replica.high.watermark.checkpoint.interval.ms is overridden to 5000
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property replica.socket.receive.buffer.bytes is overridden to 65536
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property replica.lag.max.messages is overridden to 4000
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property socket.send.buffer.bytes is overridden to 1048576
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property log.index.interval.bytes is overridden to 4096
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property producer.purgatory.purge.interval.requests is overridden to 1000
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property message.max.bytes is overridden to 1000000
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property log.flush.scheduler.interval.ms is overridden to 10000
      2013/01/21 19:21:11.103 INFO [VerifiableProperties] [main] [kafka] [] Property replica.fetch.max.bytes is overridden to 1048576
      2013/01/21 19:21:11.104 INFO [VerifiableProperties] [main] [kafka] [] Property broker.id is overridden to 466
      2013/01/21 19:21:11.104 INFO [VerifiableProperties] [main] [kafka] [] Property port is overridden to 10251
      2013/01/21 19:21:11.104 INFO [VerifiableProperties] [main] [kafka] [] Property num.network.threads is overridden to 3
      2013/01/21 19:21:11.104 INFO [VerifiableProperties] [main] [kafka] [] Property log.index.size.max.bytes is overridden to 10485760
      2013/01/21 19:21:11.104 INFO [VerifiableProperties] [main] [kafka] [] Property log.roll.hours is overridden to 168
      2013/01/21 19:21:11.105 WARN [VerifiableProperties] [main] [kafka] [] Property zk.synctime.ms is not valid
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property fetch.purgatory.purge.interval.requests is overridden to 1000
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property log.dirs is overridden to /tmp/kafka-logs
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property controller.message.queue.size is overridden to 10
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property log.retention.bytes.per.topic is overridden to
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property log.flush.interval.ms is overridden to 10000
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property log.roll.hours.per.topic is overridden to
      2013/01/21 19:21:11.105 INFO [VerifiableProperties] [main] [kafka] [] Property zk.connect is overridden to eat1-app309.corp.linkedin.com:12913,eat1-app310.corp.linkedin.com:12913,eat1-app311.corp.linkedin.com:12913,eat1-app312.corp.linkedin.com:12913,eat1-app313.corp.linkedin.com:12913/kafka-samsa
      2013/01/21 19:21:11.106 INFO [VerifiableProperties] [main] [kafka] [] Property num.io.threads is overridden to 8
      2013/01/21 19:21:11.106 INFO [VerifiableProperties] [main] [kafka] [] Property log.segment.bytes is overridden to 1073741824
      2013/01/21 19:21:11.106 INFO [VerifiableProperties] [main] [kafka] [] Property replica.socket.timeout.ms is overridden to 30000
      2013/01/21 19:21:11.106 INFO [VerifiableProperties] [main] [kafka] [] Property log.retention.hours.per.topic is overridden to
      2013/01/21 19:21:11.106 INFO [ComponentsContextLoaderListener] [main] [kafka] [] Started.
      2013/01/21 19:21:11.106 INFO [ComponentsContextLoaderListener] [main] [kafka] [] Boot sequence complete.

      Everything seems OK, but I thought I'd raise the issue anyway. I was doing something kind of evil with the brokers. I brought the entire cluster down, then rm -rf'd * in my Kafka data directory (ran out of disk space). Perhaps this triggered it? Seems kind of unrelated, give that it's ZK.

      Just raising the issue. Might not be a big deal, but better safe than sorry.

      Cheers,
      Chris

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-708 ISR becomes empty while marking a partition offline

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              nehanarkhede Neha Narkhede
              criccomini Chris Riccomini
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: