Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5544

Out of heap running CTAS against text delimited

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.10.0
    • 1.11.0
    • Storage - Parquet

    Description

      This query causes the drillbit to hang:

      create table xyz as
      select
      cast(columns[0] as bigint) l_orderkey,
      cast(columns[1] as integer) l_poartkey,
      cast(columns[2] as integer) l_suppkey,
      cast(columns[3] as integer) l_linenumber,
      cast(columns[4] as double) l_quantity,
      cast(columns[5] as double) l_extendedprice,
      cast(columns[6] as double) l_discount,
      cast(columns[7] as double) l_tax,
      cast(columns[8] as char(1)) l_returnflag,
      cast(columns[9] as char(1)) l_linestatus,
      cast(columns[10] as date) l_shipdate,
      cast(columns[11] as date) l_commitdate,
      cast(columns[12] as date) l_receiptdate,
      cast(columns[13] as char(25)) l_shipinstruct,
      cast(columns[14] as char(10)) l_shipmode,
      cast(columns[15] as varchar(44)) l_comment
      from
      `lineitem.tbl`;
      

      OOM "Java heap space" from the drillbit.log:

      drillbit.log
      ...
      2017-02-07 22:38:11,031 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:53] DEBUG o.a.d.e.s.p.ParquetDirectByteBufferAllocator - ParquetDirectByteBufferAllocator: Allocated 209715 bytes. Allocated ByteBuffer id: 1563631814
      2017-02-07 22:38:16,478 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:1] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
      java.lang.OutOfMemoryError: Java heap space
      2017-02-07 22:38:17,391 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:13] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in FragmentExecutor.
      ...
      

      To reproduce the issue please see environment details.

      Attachments

        1. dbgen.tgz
          368 kB
          Vitalii Diravka

        Issue Links

          Activity

            People

              vitalii Vitalii Diravka
              vitalii Vitalii Diravka
              Parth Chandra Parth Chandra
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: