[NIFI-6348] Allow GenerateTableFetch to order on a column without a Max-Value column

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: Extensions
• Assignee: Matt Burgess
• Reporter: Matt Burgess
• Created: 04/Jun/19 19:49
• Updated: 05/Jun/19 14:01
• Resolved: 05/Jun/19 14:00

Description

Related to NIFI-6244, if no Max-Value Column is set for GenerateTableFetch, and Partition Size > 0 (meaning paging/partitioning is done), then no column has been specified for ordering the rows. When fetching multiple pages, this causes the ordering of each fetch to be arbitrary (i.e. different ordering for each fetch), which can result in missing and/or duplicate rows. In the case of SQL Server, the parser will actually throw an error when no ORDER BY clause is provided. NIFI-6244 aimed to fix that but it was not a correct solution so was reverted.

This Jira proposes to add a "Custom ORDER BY Column" property, which will be used only if Max-Value Column is not set and Partition Size > 0. It is not required, but the documentation should explain that not using this field can result in the behavior above.

Issue Links

• links to: GitHub Pull Request #3515