Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8116

The old peers do not be deleted when the URIs of the remote process group are changed

    XMLWordPrintableJSON

    Details

      Description

      When we start transmitting for RPG, Peer Persistence will restore the peers retrieved from disk. It only checks whether the protocol has changed but it doesn't check the URIs.

       

      Due to the refactoring of NIFI-7467, peerStatusCache and getLastFetchedQueryablePeers() refer to the same object.

      We need to add a condition to check whether the URIs have been changed and discard the previous peer restored from the disk, or we will see RPG attempts to transmit data to both old and new NiFi instance and cause the errors

       

      Steps to Reproduce,

      1. create a RPG and set the URIs to nifi instance A
      2. transfer a flowfile (enforce the Peer Persistence to store the peers to disk)
      3. stop transmit and change the URIs to another instance B
      4. transfer flowfiles again

       

      2021-01-05 05:41:47,854 ERROR [Timer-Driven Process Thread-9] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=data in,targets=https://nifi-cluster-a:8443/nifi] failed to communicate with https://nifi-cluster-a:8443/nifi because the remote instance indicates that the port no longer exists
      2021-01-05 05:41:47,855 ERROR [Timer-Driven Process Thread-9] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=data in,targets=https://nifi-cluster-a:8443/nifi] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: org.apache.nifi.remote.exception.UnknownPortException: Peer[url=nifi://nifi-cluster-b:10433,CLOSED] indicates that port d106c7cb-0176-1000-0000-000072ce6099 is not known: org.apache.nifi.processor.exception.ProcessException: org.apache.nifi.remote.exception.UnknownPortException: Peer[url=nifi://nifi-cluster-b:10433,CLOSED] indicates that port d106c7cb-0176-1000-0000-000072ce6099 is not known
      org.apache.nifi.processor.exception.ProcessException: org.apache.nifi.remote.exception.UnknownPortException: Peer[url=nifi://nifi-cluster-b:10433,CLOSED] indicates that port d106c7cb-0176-1000-0000-000072ce6099 is not known
              at org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:237)
              at org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:244)
              at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
              at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
              at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.nifi.remote.exception.UnknownPortException: Peer[url=nifi://nifi-cluster-b:10433,CLOSED] indicates that port d106c7cb-0176-1000-0000-000072ce6099 is not known
              at org.apache.nifi.remote.client.socket.EndpointConnectionPool.getEndpointConnection(EndpointConnectionPool.java:258)
              at org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:127)
              at org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:223)
              ... 11 common frames omitted
      
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                s9514171 Hsin-Ying Lee
                Reporter:
                s9514171 Hsin-Ying Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m