[DRILL-8320] Prevent Infinite Pagination for Index Paginator

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.21.0
• Fix Version/s: 1.21.0
• Component/s: Storage - HTTP
• Assignee: Charles Givre
• Reporter: Charles Givre
• Created: 28/Sep/22 02:03
• Updated: 17/Oct/22 14:38
• Resolved: 17/Oct/22 14:38

Description

In some cases that use keyset/index pagination, if the API does not have a boolean column that indicates when to stop, Drill will send requests until the API stops returning data.  This PR fixes this by making the boolean parameter optional.  

If that parameter is not present, if the index result is blank or the same as the previous request, pagination will end.

Note, if the pagination parameters are buried in nested objects, this cannot be configured with a dataPath.  If the user uses a dataPath, pagination will stop at the first page.