Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Due to a bug in the DB2 driver, if next() is called twice on a result set when there are no more rows, it will throw an exception to that effect. Supposedly you can set allowNextOnExhaustedResultSet=1 on the JDBC URL or connection properties, but in my testing that has not alleviated the problem.
ExecuteSQL retrieves the entire result set at once, so the first call to next() when all rows have been fetched correctly returns false, and processing continues. However in QueryDatabaseTable, due to Max Fragments and Max Records Per Flowfile properties, it is possible that there are still rows remaining, but the current code cannot tell the difference and will try again until no rows have been fetched.
For example, in a table with 5 rows, with Max Rows Per FlowFile set to 4, the first pass will correctly process 4 rows. The second pass will process 1 and next() will return false. Then another pass is made to see if there are any more results, so the second call to next() will produce the aforementioned error.
Additional logic should be added to QueryDatabaseTable to determine whether all rows have been fetched. For example, if neither Max Fragments or Max Rows Per Flowfile have been set, then the first pass will retrieve all rows and the code does not have to check again. If Max Rows Per Flowfile has been set and the number of processed rows is less than that, then all rows have been processed and the code does not have to check again. Otherwise if exactly the "right" number of rows have been processed, then the next pass will call next() to return false, and since zero records have been processed on that pass, the code will continue as it originally did.
Attachments
Issue Links
- is duplicated by
-
NIFI-4926 QueryDatabaseTable throws SqlException after reading from DB2 table
-
- Resolved
-
- links to
