Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5950

Error updating flow version if a port is removed and another port is given the deleted port's name

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.7.1
    • Fix Version/s: 1.9.0
    • Component/s: Flow Versioning
    • Labels:
      None

      Description

      Originally reported on the users@nifi mailing list on this thread:
      https://lists.apache.org/thread.html/cf1206d65109bd35e07dfa9b7a24684a31e0580ddc2d351d2cb6eeaa@%3Cusers.nifi.apache.org%3E

      Steps to reproduce (add)

      1. Create a PG. We will call this PG1 in these instructions
      2. Add a port to PG1 named "my port"
      3. Start version control on PG1 to save to registry
      4. Create a new PG (PG2) by importing PG1 at version 1 from Registry.
      5. In PG1, delete the port named "my port"
      6. In PG1, add a port with the same name as the port you just deleted, "my port"
      7. Commit a new version of PG1 to registry
      8. Change/update PG2 to version 2. This will trigger the failure.

      Steps to reproduce (update)

      1. Create a PG. We will call this PG1 in these instructions
      2. Add a port to PG1 named "Port 1"
      3. Add a port to PG1 named "Port 2"
      4. Start version control on PG1 to save to registry
      5. Create a new PG (PG2) by importing PG1 at version 1 from Registry.
      6. In PG1, delete "Port 1"
      7. In PG1, rename "Port 2" to "Port 1"
      8. Commit a new version of PG1 to registry
      9. Change/update PG2 to version 2. This will trigger the failure.

      Analysis

      During the update, we are attempting to remove an input port and and add an input port with the same name (but different IDs). Here is an example diff that makes this apparent:

      {
        "bucketId": "898c9714-f3fa-456a-ac81-b63b7b0e7ff9",
        "componentDifferenceGroups": [
          {
            "componentId": "956f4819-77d3-3044-a6df-cf79c9be0646",
            "componentName": "my port",
            "componentType": "Input Port",
            "differences": [
              {
                "changeDescription": "Input Port was added",
                "differenceType": "COMPONENT_ADDED",
                "differenceTypeDescription": "Component Added",
                "valueB": "956f4819-77d3-3044-a6df-cf79c9be0646"
              }
            ],
            "processGroupId": "6cca3284-5caf-3555-9813-48f2dd913782"
          },
          {
            "componentId": "3434b1ce-e5be-348e-897e-cb8f79fdc42a",
            "componentName": "my port",
            "componentType": "Input Port",
            "differences": [
              {
                "changeDescription": "Input Port was removed",
                "differenceType": "COMPONENT_REMOVED",
                "differenceTypeDescription": "Component Removed",
                "valueA": "3434b1ce-e5be-348e-897e-cb8f79fdc42a"
              }
            ],
            "processGroupId": "6cca3284-5caf-3555-9813-48f2dd913782"
          }
        ],
        "flowId": "4f5a691c-428d-4439-9013-9a729b5f5d37",
        "versionA": 1,
        "versionB": 2
      }
      

      It is likely that the update flow logic in StandardProcessGroup is attempting to add the new input port before deleting the existing one. As input ports are required to have unique names, it fails with the exception/message we see in the stack trace.

      Stack trace:

      2019-01-10 21:18:38,032 ERROR [Version Control Update Thread-1] org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
      java.lang.IllegalStateException: The input port name or identifier is not available to be added.
        at org.apache.nifi.groups.StandardProcessGroup.addInputPort(StandardProcessGroup.java:512)
        at org.apache.nifi.groups.StandardProcessGroup.addInputPort(StandardProcessGroup.java:4142)
        at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3597)
        at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3367)
        at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:358)
        at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
        at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:282)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
        at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$9d72ce1d.updateProcessGroupFlow(<generated>)
        at org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:4380)
        at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
        at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:4376)
        at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
        at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:173)
        at org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.java:66)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
        at org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$f366f297.updateProcessGroupContents(<generated>)
        at org.apache.nifi.web.api.VersionsResource.updateFlowVersion(VersionsResource.java:1526)
        at org.apache.nifi.web.api.VersionsResource.lambda$null$19(VersionsResource.java:1186)
        at org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
      

      Workaround for affected versions of NiFi:

      If you are encountering this issue in your usage of NiFi, it is recommended to rename the port that is causing the issue to give it a name that has not been previously used by a deleted port, save a new version of your flow to Registry, and then do an update/change version that skips ahead to the new version of the flow with the new name. Basically, avoid deleting and adding/updating a port with the same name in a single update of the flow.

      Proposed fix:

      Change the logic in StandardProcessGroup so that when flow diffs are being applied, all COMPONENT_REMOVED changes are applied before COMPONENT_ADDED changes.

      Updated in comments below.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kdoran Kevin Doran
                Reporter:
                kdoran Kevin Doran
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 40m
                  1h 40m