Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8733

PutSQL fragmented transaction issue

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.12.1
    • None
    • NiFi Stateless
    • None
    • 3 Node Cluster

    Description

      I am hitting an issue with PutSQL processor which is pretty weird. 

      What happen is, I have a 987 flow files sharing the same attributes [fragment.identifier], [fragment.count] and their respective [fragment.index] all stuck in upstream of PutSQL.

      In PutSQL, I have configured [Support Fragmented Transactions] to TRUE. If I understand correctly, when [Support Fragmented Transactions] is TRUE, PutSQL will wait till all the fragments arrive before executing the transaction, in which in my case, the fragment all arrived however the sql execution did not.

      Only when I restart PutSQL, then all the 987 flow files will get processed successfully right away.

      If I reconfigure [Transaction Timeout] to 5000ms, then the 987 flow files will flow to failure relationship after 5000ms. This I assume PutSQL is unable to detect that all the fragments has arrived upstream and push it to failure.

      The other thing I tried is that if I reconfigure [Support Fragmented Transactions] to FALSE, then the flow files will all get processed successfully right away.

      I am not entirely sure why PutSQL is behaving this way for fragmented transaction I was hoping someone could help me out on this. 

      Attachments

        1. Screenshot 2021-06-22 at 1.48.13 AM.png
          97 kB
          ChuaSweeChin
        2. Screenshot 2021-06-22 at 1.40.37 AM.png
          174 kB
          ChuaSweeChin

        Activity

          People

            Unassigned Unassigned
            sampiochua ChuaSweeChin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: