[OAK-2931] RDBDocumentStore: mitigate effects of large query result sets

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.14, 1.2.2
• Fix Version/s: 1.0.15, 1.2.3, 1.3.0, 1.4
• Component/s: rdbmk
• Labels: resilience
• Assignee: Julian Reschke
• Reporter: Julian Reschke
• Created: 29/May/15 06:58
• Updated: 02/May/18 12:45
• Resolved: 29/May/15 13:58

Description

With the DocumentStore query API, large result sets can happen; and these are returned as List<Document>, potentially causing large amounts of memory to be allocated.

In the current implementation, the result list is generated based on a list of internal row presentations (RDBRow). These are currently freed when the method finishes. They should be freed as early as possible.

Furthermore, when the result set gets big, RDBDocumentStore should log an error containing the call chain, so that the component doing the excessive query can be identified (it should use paging instead).

(For completeness: we could also change the code to lazily populate the list; but that would be a bigger change)