Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6747

kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 0.11.0.3, 1.0.2, 1.1.1, 2.0.0
    • Component/s: streams
    • Labels:
      None

      Description

      Frederic Arno running tests against kafka-streams 1.1 and get the following stack trace (everything was working alright using kafka-streams 1.0):

      ERROR org.apache.kafka.streams.processor.internals.AssignedStreamsTasks - stream-thread [feedBuilder-XXX-StreamThread-4] Failed to close stream task, 0_2
      org.apache.kafka.common.KafkaException: TransactionalId feedBuilder-0_2: Invalid transition attempted from state READY to state ABORTING_TRANSACTION
              at org.apache.kafka.clients.producer.internals.TransactionManager.transitionTo(TransactionManager.java:757)
              at org.apache.kafka.clients.producer.internals.TransactionManager.transitionTo(TransactionManager.java:751)
              at org.apache.kafka.clients.producer.internals.TransactionManager.beginAbort(TransactionManager.java:230)
              at org.apache.kafka.clients.producer.KafkaProducer.abortTransaction(KafkaProducer.java:660)
              at org.apache.kafka.streams.processor.internals.StreamTask.closeSuspended(StreamTask.java:486)
              at org.apache.kafka.streams.processor.internals.StreamTask.close(StreamTask.java:546)
              at org.apache.kafka.streams.processor.internals.AssignedTasks.closeNonRunningTasks(AssignedTasks.java:166)
              at org.apache.kafka.streams.processor.internals.AssignedTasks.suspend(AssignedTasks.java:151)
              at org.apache.kafka.streams.processor.internals.TaskManager.suspendTasksAndState(TaskManager.java:242)
              at org.apache.kafka.streams.processor.internals.StreamThread$RebalanceListener.onPartitionsRevoked(StreamThread.java:291)
              at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare(ConsumerCoordinator.java:414)
              at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:359)
              at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:316)
              at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:290)
              at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1149)
              at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
              at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:827)
              at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:784)
              at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:750)
              at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:720)
      

      This happens when starting the same stream-processing application on 3 JVMs all running on the same linux box, JVMs are named JVM-[2-4]. All 3 instances use separate stream state.dir. No record is ever processed because the input kafka topics are empty at this stage.

      JVM-2 starts first, joined shortly after by JVM-4 and JVM-3, find the state transition logs below. The above stacktrace is from JVM-4

      [JVM-2] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      JVM-4 crashes here with above stacktrace
      [JVM-2] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from REBALANCING to ERROR
      [JVM-4] stream-client [feedBuilder-XXX] State transition from ERROR to PENDING_SHUTDOWN
      [JVM-4] stream-client [feedBuilder-XXX] State transition from PENDING_SHUTDOWN to NOT_RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from RUNNING to PENDING_SHUTDOWN
      [JVM-3] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from RUNNING to REBALANCING
      [JVM-2] stream-client [feedBuilder-XXX] State transition from PENDING_SHUTDOWN to NOT_RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from REBALANCING to RUNNING
      [JVM-3] stream-client [feedBuilder-XXX] State transition from RUNNING to PENDING_SHUTDOWN
      [JVM-4] stream-client [feedBuilder-XXX] State transition from RUNNING to PENDING_SHUTDOWN
      [JVM-3] stream-client [feedBuilder-XXX] State transition from PENDING_SHUTDOWN to NOT_RUNNING
      [JVM-4] stream-client [feedBuilder-XXX] State transition from PENDING_SHUTDOWN to NOT_RUNNING
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tedyu Zhihong Yu
                Reporter:
                frederica Frederic Arno
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: