[IMPALA-10583] Result spooling hang with unbounded spooling mem limit.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.0.0
• Component/s: Backend, Frontend
• Epic Color: ghx-label-8
• Assignee: Riza Suminto
• Reporter: Riza Suminto
• Created: 12/Mar/21 17:34
• Updated: 12/Apr/21 21:52
• Resolved: 12/Apr/21 21:52

Description

Looking at BufferedPlanRootSink and SpillableRowBatchQueue, it seems the code fail to anticipate if max_spilled_result_spooling_mem = 0 (unbounded). max_unpinned_bytes_ will be set to 0 and SpillableRowBatchQueue::IsFull() will always be true. We should change max_unpinned_bytes_ to INT64_MAX if max_spilled_result_spooling_mem = 0.

Another bug happen when setting max_result_spooling_mem = 0 (unbounded). When this happen, PlanRootSink.java will peg maxMemReservationBytes to always equal to minMemReservationBytes. I think we should override it to the default max_result_spooling_mem (100MB) or set it equal to mem_limit of the query pool.