kconsumer_1 | newly added partitions: veneer1.test.callingList51494ddbe08149a7b8ede28e39b22f21-0, veneer1.test.trEBF5974628B44491A8B6946DF72A58A8-0 kconsumer_1 | revoked partitions: kconsumer_1 | (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,912] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Adding newly assigned partitions: veneer1.test.callingList51494ddbe08149a7b8ede28e39b22f21-0, veneer1.test.trEBF5974628B44491A8B6946DF72A58A8-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,912] INFO stream-thread [split-reader-client3-StreamThread-1] State transition from RUNNING to PARTITIONS_ASSIGNED (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | [2020-02-10 00:30:39,912] INFO stream-client [split-reader-client3] State transition from RUNNING to REBALANCING (org.apache.kafka.streams.KafkaStreams) kconsumer_1 | [2020-02-10 00:30:39,920] INFO [Consumer clientId=split-reader-client3-StreamThread-1-restore-consumer, groupId=null] Unsubscribed all topics or patterns and assigned partitions (org.apache.kafka.clients.consumer.KafkaConsumer) kconsumer_1 | [2020-02-10 00:30:39,921] INFO stream-thread [split-reader-client3-StreamThread-1] partition assignment took 9 ms. kconsumer_1 | currently assigned active tasks: [4_0, 6_0, 7_0, 8_0, 9_0, 10_0] kconsumer_1 | currently assigned standby tasks: [] kconsumer_1 | revoked active tasks: [] kconsumer_1 | revoked standby tasks: [] kconsumer_1 | (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | [2020-02-10 00:30:39,922] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Found no committed offset for partition veneer1.test.trEBF5974628B44491A8B6946DF72A58A8-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,922] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Found no committed offset for partition veneer1.test.callingList51494ddbe08149a7b8ede28e39b22f21-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,925] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Resetting offset for partition veneer1.test.trEBF5974628B44491A8B6946DF72A58A8-0 to offset 0. (org.apache.kafka.clients.consumer.internals.SubscriptionState) kconsumer_1 | [2020-02-10 00:30:39,925] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Resetting offset for partition veneer1.test.callingList51494ddbe08149a7b8ede28e39b22f21-0 to offset 0. (org.apache.kafka.clients.consumer.internals.SubscriptionState) kconsumer_1 | [2020-02-10 00:30:39,979] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Found no committed offset for partition veneer1.test.callingList51494ddbe08149a7b8ede28e39b22f21-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,982] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Found no committed offset for partition veneer1.test.trCE69B90D8C334021AB4F67424FD1FBF8-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,982] INFO [Consumer clientId=split-reader-client3-StreamThread-1-consumer, groupId=split-reader3] Found no committed offset for partition veneer1.test.trEBF5974628B44491A8B6946DF72A58A8-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) kconsumer_1 | [2020-02-10 00:30:39,987] ERROR stream-thread [split-reader-client3-StreamThread-1] Encountered the following unexpected Kafka exception during processing, this usually indicate Streams internal errors: (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | org.apache.kafka.streams.errors.ProcessorStateException: Error opening store KSTREAM-REDUCE-STATE-STORE-0000000032 at location /tmp/split-reader3/10_0/rocksdb/KSTREAM-REDUCE-STATE-STORE-0000000032 kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:87) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:191) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:228) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:44) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:58) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:208) kconsumer_1 | at org.apache.kafka.streams.processor.internals.AbstractTask.registerStateStores(AbstractTask.java:210) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamTask.initializeStateStores(StreamTask.java:323) kconsumer_1 | at org.apache.kafka.streams.processor.internals.AssignedTasks.initializeNewTasks(AssignedTasks.java:76) kconsumer_1 | at org.apache.kafka.streams.processor.internals.TaskManager.updateNewAndRestoringTasks(TaskManager.java:385) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:769) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:698) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:671) kconsumer_1 | Caused by: org.rocksdb.RocksDBException: lock : /tmp/split-reader3/10_0/rocksdb/KSTREAM-REDUCE-STATE-STORE-0000000032/LOCK: No locks available kconsumer_1 | at org.rocksdb.RocksDB.open(Native Method) kconsumer_1 | at org.rocksdb.RocksDB.open(RocksDB.java:286) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:75) kconsumer_1 | ... 15 more kconsumer_1 | [2020-02-10 00:30:39,992] INFO stream-thread [split-reader-client3-StreamThread-1] State transition from PARTITIONS_ASSIGNED to PENDING_SHUTDOWN (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | [2020-02-10 00:30:39,992] INFO stream-thread [split-reader-client3-StreamThread-1] Shutting down (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | [2020-02-10 00:30:40,024] INFO [Consumer clientId=split-reader-client3-StreamThread-1-restore-consumer, groupId=null] Unsubscribed all topics or patterns and assigned partitions (org.apache.kafka.clients.consumer.KafkaConsumer) kconsumer_1 | [2020-02-10 00:30:40,025] INFO [Producer clientId=split-reader-client3-StreamThread-1-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer) kconsumer_1 | [2020-02-10 00:30:40,035] INFO stream-thread [split-reader-client3-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | [2020-02-10 00:30:40,035] INFO stream-client [split-reader-client3] State transition from REBALANCING to ERROR (org.apache.kafka.streams.KafkaStreams) kconsumer_1 | [2020-02-10 00:30:40,035] ERROR stream-client [split-reader-client3] All stream threads have died. The instance will be in error state and should be closed. (org.apache.kafka.streams.KafkaStreams) kconsumer_1 | [2020-02-10 00:30:40,035] INFO stream-thread [split-reader-client3-StreamThread-1] Shutdown complete (org.apache.kafka.streams.processor.internals.StreamThread) kconsumer_1 | Exception in thread "split-reader-client3-StreamThread-1" org.apache.kafka.streams.errors.ProcessorStateException: Error opening store KSTREAM-REDUCE-STATE-STORE-0000000032 at location /tmp/split-reader3/10_0/rocksdb/KSTREAM-REDUCE-STATE-STORE-0000000032 kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:87) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:191) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:228) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:44) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:58) kconsumer_1 | at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48) kconsumer_1 | at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:208) kconsumer_1 | at org.apache.kafka.streams.processor.internals.AbstractTask.registerStateStores(AbstractTask.java:210) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamTask.initializeStateStores(StreamTask.java:323) kconsumer_1 | at org.apache.kafka.streams.processor.internals.AssignedTasks.initializeNewTasks(AssignedTasks.java:76) kconsumer_1 | at org.apache.kafka.streams.processor.internals.TaskManager.updateNewAndRestoringTasks(TaskManager.java:385) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:769) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:698) kconsumer_1 | at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:671) kconsumer_1 | Caused by: org.rocksdb.RocksDBException: lock : /tmp/split-reader3/10_0/rocksdb/KSTREAM-REDUCE-STATE-STORE-0000000032/LOCK: No locks available kconsumer_1 | at org.rocksdb.RocksDB.open(Native Method) kconsumer_1 | at org.rocksdb.RocksDB.open(RocksDB.java:286) kconsumer_1 | at org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:75) kconsumer_1 | ... 15 more kconsumer_1 | [2020-02-10 02:05:39,076] INFO stream-client [split-reader-client3] State transition from ERROR to PENDING_SHUTDOWN (org.apache.kafka.streams.KafkaStreams)