[DRILL-3770] Query with window function having just ORDER BY clause runs out of memory on large datasets

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.3.0
• Component/s: Execution - Flow
• Labels: window_function

Environment

10 Performance Nodes
DRILL_MAX_DIRECT_MEMORY=100g
DRILL_INIT_HEAP="8g"
DRILL_MAX_HEAP="8g"
planner.memory.query_max_memory_per_node bumped up to 20 GB
TPC-DS SF 1000 dataset (Parquet)

• Assignee: Abdel Hakim Deneche
• Reporter: Abhishek Girish
• Reviewer: Abhishek Girish
• Created: 12/Sep/15 01:15
• Updated: 12/Dec/15 01:38
• Resolved: 10/Nov/15 22:59

Description

The following query runs out of memory:

SELECT SUM(ss.ss_net_paid_inc_tax) OVER (ORDER BY ss.ss_store_sk) FROM store_sales ss LIMIT 20;
java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes ran out of memory while executing the query.

Fragment 0:0

[Error Id: 9c441211-65ec-4206-9e6b-d6ae9c2903be on ucs-node6.perf.lab:31010]
	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
	at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
	at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
	at sqlline.SqlLine.print(SqlLine.java:1583)
	at sqlline.Commands.execute(Commands.java:852)
	at sqlline.Commands.sql(Commands.java:751)
	at sqlline.SqlLine.dispatch(SqlLine.java:738)
	at sqlline.SqlLine.begin(SqlLine.java:612)
	at sqlline.SqlLine.start(SqlLine.java:366)
	at sqlline.SqlLine.main(SqlLine.java:259)

Issue Links

• is part of: DRILL-3952 Improve Window Functions performance when not all batches are required to process the current batch