Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6254

IllegalArgumentException: the requested size must be non-negative

    Details

      Description

      Flatten query fails due to IllegalArgumentException: the requested size must be non-negative.

      Script to generate JSON data file is attached here.

      0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_all_types_jsn_to_parquet AS 
      . . . . . . . . . . . . . . > SELECT
      . . . . . . . . . . . . . . > CAST( col_int AS INT) col_int, 
      . . . . . . . . . . . . . . > CAST( col_bigint AS BIGINT) col_bigint, 
      . . . . . . . . . . . . . . > CAST( col_char AS CHAR(10)) col_char, 
      . . . . . . . . . . . . . . > CAST( col_fxdln_str AS VARCHAR(256)) col_fxdln_str, 
      . . . . . . . . . . . . . . > CAST( col_varln_str AS VARCHAR(256)) col_varln_str, 
      . . . . . . . . . . . . . . > CAST( col_float AS FLOAT) col_float, 
      . . . . . . . . . . . . . . > CAST( col_double AS DOUBLE PRECISION) col_double, 
      . . . . . . . . . . . . . . > CAST( col_date AS DATE) col_date, 
      . . . . . . . . . . . . . . > CAST( col_time AS TIME) col_time, 
      . . . . . . . . . . . . . . > CAST( col_tmstmp AS TIMESTAMP) col_tmstmp, 
      . . . . . . . . . . . . . . > CAST( col_boolean AS BOOLEAN) col_boolean, 
      . . . . . . . . . . . . . . > col_binary, 
      . . . . . . . . . . . . . . > array_of_ints from `all_supported_types.json`;
      +-----------+----------------------------+
      | Fragment | Number of records written |
      +-----------+----------------------------+
      | 0_0 | 9 |
      +-----------+----------------------------+
      1 row selected (0.29 seconds)
      

      Reset all options and set slice_target=1
      alter system reset all;
      alter system set `planner.slice_target`=1;

      output_batch_size was set to its default value
      drill.exec.memory.operator.output_batch_size = 16777216

       

      select *, flatten(array_of_ints) from tbl_all_types_jsn_to_parquet;
      
      Error: SYSTEM ERROR: IllegalArgumentException: the requested size must be non-negative
      
      Fragment 0:0
      
      [Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010] (state=,code=0)
      0: jdbc:drill:schema=dfs.tmp>
      

      Stack trace from drillbit.log

      2018-03-15 12:19:43,916 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 255538af-bcd5-98ee-32e0-68d98fc4a6fa: select *, flatten(array_of_ints) from tbl_all_types_jsn_to_parquet
      2018-03-15 12:19:43,952 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, numFiles: 1
      2018-03-15 12:19:43,953 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, numFiles: 1
      2018-03-15 12:19:43,966 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.parquet.Metadata - Took 0 ms to get file statuses
      2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Time: 2ms total, 2.927366ms avg, 2ms max.
      2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Earliest start: 2.829000 μs, Latest start: 2.829000 μs, Average start: 2.829000 μs .
      2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO o.a.d.exec.store.parquet.Metadata - Took 3 ms to read file metadata
      2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
      2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: State to report: RUNNING
      2018-03-15 12:19:44,905 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: State change requested RUNNING --> FAILED
      2018-03-15 12:19:44,927 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: State change requested FAILED --> FINISHED
      2018-03-15 12:19:44,928 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalArgumentException: the requested size must be non-negative
      
      Fragment 0:0
      
      [Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
      org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalArgumentException: the requested size must be non-negative
      
      Fragment 0:0
      
      [Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
       at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:300) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
       at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
      Caused by: java.lang.IllegalArgumentException: the requested size must be non-negative
       at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) ~[guava-18.0.jar:na]
       at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:224) ~[drill-memory-base-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:211) ~[drill-memory-base-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.vector.BigIntVector.allocateBytes(BigIntVector.java:239) ~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.vector.BigIntVector.allocateNew(BigIntVector.java:219) ~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.vector.RepeatedBigIntVector.allocateNew(RepeatedBigIntVector.java:272) ~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.vector.AllocationHelper.allocatePrecomputedChildCount(AllocationHelper.java:41) ~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:66) ~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.RecordBatchSizer$ColumnSize.allocateVector(RecordBatchSizer.java:403) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.doAlloc(FlattenRecordBatch.java:276) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.handleRemainder(FlattenRecordBatch.java:240) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext(FlattenRecordBatch.java:165) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:233) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226) ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_161]
       at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_161]
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595) ~[hadoop-common-2.7.0-mapr-1707.jar:na]
       at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
       ... 4 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ppenumarthy Padma Penumarthy
                Reporter:
                khfaraaz Khurram Faraaz
                Reviewer:
                Karthikeyan Manivannan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: