Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Duplicate
-
1.3.0
-
None
-
None
-
None
-
Linux
-
Important
Description
hello,
When i start one consumeKafka_0_10 processor ,all of the processors are all right.
but when i start consumeKafka_0_10 processor more than eleven,all of the processors get into an indefinite stuck state.
like in the picture ,look like everything is all right, but all processor(even processor is not consumeKafka_0_10 ) are stop work.All of the flowfile don't move.
And when i stop them , some of consumeKafka_0_10 processor are warning
Was interrupted while trying to communicate with Kafka with lease {}. Will roll back session and discard any partially received data.
in the nifi-app.log
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.SplitJson 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@a987d5e // Another save pending = false 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784. Will roll back session and discard any partially received data. 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
no Kerberos involved
All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group ID'
The configuration file has been uploaded in the attachment 2.png
the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'
one consumeKafka processor ----------one topic----------one partition --------------one replication-factor
Every consumeKafka processor has a different topic.
And i had tried out nifi 1.4.0,but it can not resolved the issues.
thanks
Sorry,my english is so bad.Please forgive me.
Attachments
Attachments
Issue Links
- duplicates
-
NIFI-4670 when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state
- Resolved