Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3770

Query with window function having just ORDER BY clause runs out of memory on large datasets

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.2.0
    • 1.3.0
    • Execution - Flow
    • 10 Performance Nodes
      DRILL_MAX_DIRECT_MEMORY=100g
      DRILL_INIT_HEAP="8g"
      DRILL_MAX_HEAP="8g"
      planner.memory.query_max_memory_per_node bumped up to 20 GB
      TPC-DS SF 1000 dataset (Parquet)

    Description

      The following query runs out of memory:

      SELECT SUM(ss.ss_net_paid_inc_tax) OVER (ORDER BY ss.ss_store_sk) FROM store_sales ss LIMIT 20;
      java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes ran out of memory while executing the query.
      
      Fragment 0:0
      
      [Error Id: 9c441211-65ec-4206-9e6b-d6ae9c2903be on ucs-node6.perf.lab:31010]
      	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
      	at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
      	at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
      	at sqlline.SqlLine.print(SqlLine.java:1583)
      	at sqlline.Commands.execute(Commands.java:852)
      	at sqlline.Commands.sql(Commands.java:751)
      	at sqlline.SqlLine.dispatch(SqlLine.java:738)
      	at sqlline.SqlLine.begin(SqlLine.java:612)
      	at sqlline.SqlLine.start(SqlLine.java:366)
      	at sqlline.SqlLine.main(SqlLine.java:259)
      

      Attachments

        1. profile.json
          531 kB
          Abhishek Girish
        2. drillbit.log.txt
          13 kB
          Abhishek Girish

        Issue Links

          Activity

            People

              adeneche Abdel Hakim Deneche
              agirish Abhishek Girish
              Abhishek Girish Abhishek Girish
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: