[NIFI-11192] If Port moved from parent to child group or vice versa between flow versions, version change can leave nifi in bad state - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Blocker
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.0.0-M1, 1.21.0
Component/s: Core Framework
Labels:
None

Description

To reproduce:

Create a Process Group, Parent.

Inside of Parent, create:

A processor, for example, ReplaceText
A Process Group, Child.

Inside of Child, create:

An Input Port, In
A Processor, for example, UpdateAttribute
A connection between the two components

Then connect ReplaceText to the Input Port, In.

Save the Parent PG flow as Version 1 of a flow.

Now, create a new Processor, say RouteOnAttribute, within the Parent PG.

Move the destination of the connection from the Input Port to RouteOnAttribute.

Step into the Child PG. Select all components, right-click, and choose "Move to Parent"

Save Parent PG as Version 2 of the flow.

Now, attempt to Change Version on the Parent Group. Change the version to Version 1.

The version change will fail with an error: "Failed to perform update flow request due to 42fb2904-c774-359b-5368-2e48b60ac02d is the destination of another component" and the logs will have a stack trace:

2023-02-16 15:18:28,830 ERROR [Process Group Update Thread-1] o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request
java.lang.IllegalStateException: 42fb2904-c774-359b-5368-2e48b60ac02d is the destination of another component
    at org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:562)
    at org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:542)
    at org.apache.nifi.groups.StandardProcessGroup.removeInputPort(StandardProcessGroup.java:637)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingComponents(StandardVersionedComponentSynchronizer.java:948)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingInputPorts(StandardVersionedComponentSynchronizer.java:873)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:410)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260)
    at org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:556)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255)
    at org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972)
    at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3952)
    at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
    at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:308)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
    at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$2edbbc7e.updateProcessGroupFlow(<generated>)
    at org.apache.nifi.web.StandardNiFiServiceFacade$13.update(StandardNiFiServiceFacade.java:5848)
    at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:130)
    at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5843)
    at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
    at org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.java:66)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
    at org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$cd414c69.updateProcessGroupContents(<generated>)
    at org.apache.nifi.web.api.VersionsResource.performUpdateFlow(VersionsResource.java:1221)
    at org.apache.nifi.web.api.VersionsResource.performUpdateFlow(VersionsResource.java:85)
    at org.apache.nifi.web.api.FlowUpdateResource.updateFlow(FlowUpdateResource.java:428)
    at org.apache.nifi.web.api.FlowUpdateResource.lambda$submitFlowUpdateRequest$5(FlowUpdateResource.java:281)
    at org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

This now leaves NiFi in a bad state, where the flow is partially updated but not fully. What's worse, though, is that it leaves the ReplaceText Processor connected to an Input Port within the same Process Group, which is not legal.

At this point, the flow has not be written to the flow.json.gz/flow.xml.gz because the update operation failed.

Now, move the ReplaceText Processor to the side a bit, in order to trigger the flow to be written out again.

Now, attempt to restart NiFI, and it will fail to startup, yielding the following error in the logs:

2023-02-16 15:23:10,770 WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup of context o.e.j.w.WebAppContext@3fe8ad3f{nifi-api,/nifi-api,file:///Users/mpayne/devel/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/work/jetty/nifi-web-api-2.0.0-SNAPSHOT.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-2.0.0-SNAPSHOT.war}
org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalStateException: Cannot add Connection from PROCESSOR[7a6913fa-3a97-3ce2-c128-d915258fae47] from Process Group [5bd874fd-0186-1000-222a-d40ae014c2d3] to Process Group [5bd874fd-0186-1000-222a-d40ae014c2d3] because its destination [42fb2904-c774-359b-5368-2e48b60ac02d] is an Input Port but the Input Port does not belong to a child Process Group
    at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:448)
    at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:206)
    at org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
    at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1638)
    at org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
    at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:817)
    at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538)
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
    at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
    at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.server.Server.start(Server.java:423)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.server.Server.doStart(Server.java:387)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:815)
    at org.apache.nifi.NiFi.<init>(NiFi.java:172)
    at org.apache.nifi.NiFi.<init>(NiFi.java:83)
    at org.apache.nifi.NiFi.main(NiFi.java:332)
Caused by: java.lang.IllegalStateException: Cannot add Connection from PROCESSOR[7a6913fa-3a97-3ce2-c128-d915258fae47] from Process Group [5bd874fd-0186-1000-222a-d40ae014c2d3] to Process Group [5bd874fd-0186-1000-222a-d40ae014c2d3] because its destination [42fb2904-c774-359b-5368-2e48b60ac02d] is an Input Port but the Input Port does not belong to a child Process Group
    at org.apache.nifi.groups.StandardProcessGroup.addConnection(StandardProcessGroup.java:1310)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addConnection(StandardVersionedComponentSynchronizer.java:3312)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeConnections(StandardVersionedComponentSynchronizer.java:661)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:461)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1149)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:531)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:417)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260)
    at org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:556)
    at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255)
    at org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972)
    at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:439)
    ... 45 common frames omitted

There are two issues here:

We should not fail to update the version of the flow. We need to fix the logic in order to properly handle the moving of a port between groups.
If for some reason, we do fail to perform a Change Version request, we currently leave the flow in a bad state. We should instead revert all changes to the flow for which the change was attempted. This should prevent us from getting the flow into a bad state, even if an update fails.

Attachments

Issue Links

relates to

NIFI-11251 Nifi Registry Client with Nested PG - sync issue

Resolved

links to

GitHub Pull Request #6981

Activity

People

Assignee:: Mark Payne

Reporter:: Mark Payne

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 16/Feb/23 20:25

Updated:: 28/Mar/23 16:04

Resolved:: 22/Feb/23 20:46

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

40m