[NIFI-8733] PutSQL fragmented transaction issue

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.12.1
• Component/s: NiFi Stateless
• Environment: 3 Node Cluster
• Assignee: Unassigned
• Reporter: ChuaSweeChin
• Created: 24/Jun/21 02:50
• Updated: 15/Jul/21 18:28

Description

I am hitting an issue with PutSQL processor which is pretty weird. 

What happen is, I have a 987 flow files sharing the same attributes [fragment.identifier], [fragment.count] and their respective [fragment.index] all stuck in upstream of PutSQL.

In PutSQL, I have configured [Support Fragmented Transactions] to TRUE. If I understand correctly, when [Support Fragmented Transactions] is TRUE, PutSQL will wait till all the fragments arrive before executing the transaction, in which in my case, the fragment all arrived however the sql execution did not.

Only when I restart PutSQL, then all the 987 flow files will get processed successfully right away.

If I reconfigure [Transaction Timeout] to 5000ms, then the 987 flow files will flow to failure relationship after 5000ms. This I assume PutSQL is unable to detect that all the fragments has arrived upstream and push it to failure.

The other thing I tried is that if I reconfigure [Support Fragmented Transactions] to FALSE, then the flow files will all get processed successfully right away.

I am not entirely sure why PutSQL is behaving this way for fragmented transaction I was hoping someone could help me out on this.