Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8733

PutSQL fragmented transaction issue

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.12.1
    • Fix Version/s: None
    • Component/s: NiFi Stateless
    • Labels:
      None
    • Environment:
      3 Node Cluster

      Description

      I am hitting an issue with PutSQL processor which is pretty weird. 

      What happen is, I have a 987 flow files sharing the same attributes [fragment.identifier], [fragment.count] and their respective [fragment.index] all stuck in upstream of PutSQL.

      In PutSQL, I have configured [Support Fragmented Transactions] to TRUE. If I understand correctly, when [Support Fragmented Transactions] is TRUE, PutSQL will wait till all the fragments arrive before executing the transaction, in which in my case, the fragment all arrived however the sql execution did not.

      Only when I restart PutSQL, then all the 987 flow files will get processed successfully right away.

      If I reconfigure [Transaction Timeout] to 5000ms, then the 987 flow files will flow to failure relationship after 5000ms. This I assume PutSQL is unable to detect that all the fragments has arrived upstream and push it to failure.

      The other thing I tried is that if I reconfigure [Support Fragmented Transactions] to FALSE, then the flow files will all get processed successfully right away.

      I am not entirely sure why PutSQL is behaving this way for fragmented transaction I was hoping someone could help me out on this. 

        Attachments

        1. Screenshot 2021-06-22 at 1.40.37 AM.png
          174 kB
          ChuaSweeChin
        2. Screenshot 2021-06-22 at 1.48.13 AM.png
          97 kB
          ChuaSweeChin

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sampiochua ChuaSweeChin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: