[NIFI-4682] when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 1.3.0
• Environment: Linux
• Flags: Important
• Assignee: Unassigned
• Reporter: clearlove
• Created: 09/Dec/17 00:42
• Updated: 13/Jan/18 15:39
• Resolved: 13/Jan/18 15:39

Description

hello,
When i start one consumeKafka_0_10 processor ,all of the processors are all right.
but when i start consumeKafka_0_10 processor more than eleven,all of the processors get into an indefinite stuck state.

like in the picture ,look like everything is all right, but all processor(even processor is not consumeKafka_0_10 ) are stop work.All of the flowfile don't move.
And when i stop them , some of consumeKafka_0_10 processor are warning
Was interrupted while trying to communicate with Kafka with lease {}. Will roll back session and discard any partially received data.

in the nifi-app.log

2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.SplitJson
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@a987d5e // Another save pending = false
2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784. Will roll back session and discard any partially received data.
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run

no Kerberos involved
All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group ID'
The configuration file has been uploaded in the attachment 2.png
the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'

one consumeKafka processor ----------one topic----------one partition --------------one replication-factor
Every consumeKafka processor has a different topic.
And i had tried out nifi 1.4.0,but it can not resolved the issues.

thanks

Sorry,my english is so bad.Please forgive me.

Issue Links

• duplicates: NIFI-4670 when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state