Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5544

Out of heap running CTAS against text delimited

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.11.0
    • Component/s: Storage - Parquet
    • Labels:
    • Environment:

      Description

      This query causes the drillbit to hang:

      create table xyz as
      select
      cast(columns[0] as bigint) l_orderkey,
      cast(columns[1] as integer) l_poartkey,
      cast(columns[2] as integer) l_suppkey,
      cast(columns[3] as integer) l_linenumber,
      cast(columns[4] as double) l_quantity,
      cast(columns[5] as double) l_extendedprice,
      cast(columns[6] as double) l_discount,
      cast(columns[7] as double) l_tax,
      cast(columns[8] as char(1)) l_returnflag,
      cast(columns[9] as char(1)) l_linestatus,
      cast(columns[10] as date) l_shipdate,
      cast(columns[11] as date) l_commitdate,
      cast(columns[12] as date) l_receiptdate,
      cast(columns[13] as char(25)) l_shipinstruct,
      cast(columns[14] as char(10)) l_shipmode,
      cast(columns[15] as varchar(44)) l_comment
      from
      `lineitem.tbl`;
      

      OOM "Java heap space" from the drillbit.log:

      drillbit.log
      ...
      2017-02-07 22:38:11,031 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:53] DEBUG o.a.d.e.s.p.ParquetDirectByteBufferAllocator - ParquetDirectByteBufferAllocator: Allocated 209715 bytes. Allocated ByteBuffer id: 1563631814
      2017-02-07 22:38:16,478 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:1] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
      java.lang.OutOfMemoryError: Java heap space
      2017-02-07 22:38:17,391 [2765b496-0b5b-a3df-c252-a8bb9cd2e52f:frag:1:13] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in FragmentExecutor.
      ...
      

      To reproduce the issue please see environment details.

        Attachments

        1. dbgen.tgz
          368 kB
          Vitalii Diravka

          Issue Links

            Activity

              People

              • Assignee:
                vitalii Vitalii Diravka
                Reporter:
                vitalii Vitalii Diravka
                Reviewer:
                Parth Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: