Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-13927

PublishGCPPubSub processor stop working and is stucked when using Record Oriented mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.26.0, 2.0.0-M4
    • 2.0.0, 1.29.0, 1.28.1
    • Extensions
    • None

    Description

      When using the PublishGCPPubSub processor with the Record Oriented mode the processor will stay stucked. It won't processor any FlowFile. When the processor is terminated the thread isn't released by the processor.

      It FlowFile Oriented mode, it works fine.

      It seems to be linked to how many records are in the FlowFile, becase when having a single record in the FlowFile, it seems to have no issue (may be it take more time to appear).

      But if you start to have more records in the FlowFile, it won't work. To make the processor works again you either need to restart the node or removing and recreating the processor. But it's temporary, because it some point it will be stucked again. When stucked the processor won't be unstucked even after days (we had a processor stucked for 3 days straight).

      You can find attached a thread dump when the processor was running and stucked, and with ~10 threads not released . And you can also find an example that will show the issue with a FlowFile of 500 records that instantly stucked in the publish.

      Attachments

        1. nifi.tdump
          378 kB
          Julien G.
        2. EXAMPLE.json
          28 kB
          Julien G.

        Activity

          People

            pvillard Pierre Villard
            juldrixx Julien G.
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h
                2h