[DRILL-5294] Managed External Sort throws an OOM during the merge and spill phase

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: Execution - Relational Operators
• Assignee: Paul Rogers
• Reporter: Rahul Kumar Challapalli
• Created: 23/Feb/17 00:51
• Updated: 21/Mar/17 18:38
• Resolved: 02/Mar/17 23:39

Description

commit # : 38f816a45924654efd085bf7f1da7d97a4a51e38

The below query fails with managed sort while it succeeds on the old sort

select * from (select columns[433] col433, columns[0], columns[1],columns[2],columns[3],columns[4],columns[5],columns[6],columns[7],columns[8],columns[9],columns[10],columns[11] from dfs.`/drill/testdata/resource-manager/3500cols.tbl` order by columns[450],columns[330],columns[230],columns[220],columns[110],columns[90],columns[80],columns[70],columns[40],columns[10],columns[20],columns[30],columns[40],columns[50]) d where d.col433 = 'sjka skjf';
Error: RESOURCE ERROR: External Sort encountered an error while spilling to disk

Fragment 1:11

[Error Id: 0aa20284-cfcc-450f-89b3-645c280f33a4 on qa-node190.qa.lab:31010] (state=,code=0)

Env :

No of Drillbits : 1
DRILL_MAX_DIRECT_MEMORY="32G"
DRILL_MAX_HEAP="4G"

Attached the logs and profile. Data is too large for a jira