Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8652

PutAzureCosmosDBRecord generates Long running task detected on processor warning and does not update 'In' process rate.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.13.2
    • Fix Version/s: None
    • Component/s: Extensions
    • Labels:

      Description

      When we generate and insert bulk records into Azure Cosmos DB with PutAzureCosmosDBRecord, we don't see update of the 'In' process rate in the processor and see the warning, while we see the corresponding records in Azure Cosmos DB side.  I captured the screenshot in the below attachment. 

       

      For now, 2 workarounds I found to overcome this issue:

      1. Change the consistency property to 'Eventual'. 
      2. If you keep consistency as 'Session', add a success relationship/connection to a dummy log processor. 

       

      The related warning in the log file:

       

      2021-06-02 16:41:39,395 WARN [Long Running Task Monitor Thread-1] o.a.n.c.monitor.LongRunningTaskMonitor Long running task detected on processor [id=256082ee-341e-3c42-e379-5c5d365e9e18, name=PutAzureCosmosDBRecord, type=PutAzureCosmosDBRecord]. Task time: 324 seconds. Stack trace:
      "Timer-Driven Process Thread-5" Id=74 WAITING on java.util.concurrent.CountDownLatch$Sync@307e6f8b
      at java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
      at java.base@11.0.11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
      at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
      at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
      at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
      at java.base@11.0.11/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
      at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
      at reactor.core.publisher.Mono.block(Mono.java:1678)
      at com.azure.cosmos.CosmosContainer.blockItemResponse(CosmosContainer.java:210)
      at com.azure.cosmos.CosmosContainer.createItem(CosmosContainer.java:144)
      at org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.bulkInsert(PutAzureCosmosDBRecord.java:134)
      at org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecord.onTrigger(PutAzureCosmosDBRecord.java:192)
      at app//org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1180)
      at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
      at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
      at java.base@11.0.11/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      at java.base@11.0.11/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
      at java.base@11.0.11/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
      at java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      at java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
      Number of Locked Synchronizers: 1

      • java.util.concurrent.ThreadPoolExecutor$Worker@52bef30b

        Attachments

        1. PutAzureCosmosDBRecordWarning.jpg
          390 kB
          Seokwon Yang

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                sjyang18 Seokwon Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m